Re: Home NAS
What can be newer or not existent yesterday, but has the same filename? Something that one changed with an editor? Would not be better to use a version contro system? Rod. On Mon, 18 Nov 2019, Nick Holland wrote: On 2019-11-17 11:39, Jean-François Simon wrote: Hi, I found it, there exist glastree which is available from ports. Nice small "poor man's" backup as the author qualifies, though makes incremental backup through hard links: # if yesterday does not exist or today is newer, copy the file # else hard link the file to yesterday rsync --link-dest -- it's been in rsync for well over 10 years at this point. Little wrapper shell script and away you go...
Re: Home NAS
On 2019-11-17 11:39, Jean-François Simon wrote: > Hi, > > I found it, there exist glastree which is available from ports. > > Nice small "poor man's" backup as the author qualifies, > though makes incremental backup through hard links: > > # if yesterday does not exist or today is newer, copy the file > # else hard link the file to yesterday rsync --link-dest -- it's been in rsync for well over 10 years at this point. Little wrapper shell script and away you go... Nick.
Re: Home NAS
Den lör 16 nov. 2019 kl 22:49 skrev Karel Gardas : > > I tried a home NAS with ZFS, then BTRFS. Those filesystems needs tons of > RAM (~1 GB of RAM by TB of disk), preferably ECC. > > For NAS you prefer ECC anyway and 1 GB RAM consumption per 1 TB of drive > is urban legend probably passed by folks using deduplication. Or people who do not want to swap while doing extensive fsck of huge partitions with lots of small files in them. Most recommendations are based on all corner cases and not just the happy-case when you stash a single movie on a nas over the home network. Yes, dedup uses ram most of the time if it can, but other things do too. Also, "excess" ram in these cases turn into read caches so its not lost on you either. -- May the most significant bit of your life be positive.
Re: Home NAS
Hi, On 11/17, Predrag Punosevac wrote: > Patrick Marchand wrote: > > On 11/15, Predrag Punosevac wrote: > > > Patrick Marchand wrote: > > > > I'll be playing around with DragonflyBSD Hammer2 (and multiple offsite > > > > backups) for a home NAS over the next few weeks. I'll probably do a > > > > presentation about the experience at the Montreal BSD user group > > > > afterwards. It does not require as many ressources as ZFS or BTRFS, > > > > but offers many similar features. > > > > > > > > > > Been there, done that! > > Cool ! I might ping you off-list with questions when I get to it. > > > > Any time. Either this private email or at my work predr...@cs.cmu.edu > I wish I was a bit closer to Montreal to come to your monthly meeting. I > love Quebec and Montreal in particular. Thanks ! > > I'm not planning on using jails much, instead I'll be using the > > DFly NFS with OpenBSD to experiment with virtualization. > > > > > I am not sure that I am following. How is DF NFS server related to > OpenBSD (if I understand correctly) virtualization. Are you trying to > store OpenBSD vmm images on the NFS share exported from a DF server? > That is a really, really bad idea. > > > https://marc.info/?l=dragonfly-users=140384130921709=2 MirageOS and PXE booted OpenBSD is what I really want to play with, but well see as I go along, breaking stuff is kind of the idea here. > > > > DragonFly which gets it software RAID discipline through old > > > unmaintained FreeBSD natacontrol utility. Hardware RAID cards are not > > > frequently tested and community seems to be keen on treating DF as a > > > desktop OS rather than a storage workhorse. Having said that HDD are > > > cheap this days and home users probably don't need anything bigger than > > > a 12TB mirror. > > I dont store much anyways, so I'll see as I go. > > > > 12 TB is the sweet spot when it comes GB/dollar for platter HDDs. As it's more for an experiment and maybe some bsd systems work and it will be running in my room, I started with a 1TB ssd.
Re: Home NAS
Milun Rajkovic wrote: > Pardon my ignorance and lack of deeper knowledge regarding the matter, > but since when is XFS not even considered for such uses? > Since 2005 if you are Solaris guy. Since 2008 if you are ZFS on FreeBSD or Hammer 1 DragonFly guy. XFS is indeed the most stable and reliable file system for Linux and in principle there is nothing wrong with using XFS on the top of hardware or software RAID if you don't care about data integrity, self-healing, COW, snapshots, replication and similar things. If you put LVM2 between the RAID and XFS you could theoretically get snapshots of logical volumes and perhaps even restore something. However LVM2 snapshots are expensive and not really practical contrary to Red Hat PR debarment claims. Hopefully some of old Irix SGI who are lurking on this mailing list could tell you more things I don't know or I forgot since my old Irix days. Pozdrav, Pedja > Cheers > Milun
Re: Home NAS
Patrick Marchand wrote: > Hello, > > On 11/15, Predrag Punosevac wrote: > > Patrick Marchand wrote: > > > I'll be playing around with DragonflyBSD Hammer2 (and multiple offsite > > > backups) for a home NAS over the next few weeks. I'll probably do a > > > presentation about the experience at the Montreal BSD user group > > > afterwards. It does not require as many ressources as ZFS or BTRFS, > > > but offers many similar features. > > > > > > > Been there, done that! > Cool ! I might ping you off-list with questions when I get to it. > Any time. Either this private email or at my work predr...@cs.cmu.edu I wish I was a bit closer to Montreal to come to your monthly meeting. I love Quebec and Montreal in particular. > > H2 lacks built in backup mechanism. I was hoping that H2 will get some > > kind "hammer mirror-copy" of H1, or "zfs send/receive". My server is > > still on H1 and I really enjoy being able to continuously back it up. > > That's the only thing I am missing in H2. On the positive note H2 did > > get support for boot environment manager last year. > > > > https://github.com/newnix/dfbeadm > > > > Also DF jails are stuck in 2004 or something like that. I like their > > NFSv3. > I'm not planning on using jails much, instead I'll be using the > DFly NFS with OpenBSD to experiment with virtualization. > I am not sure that I am following. How is DF NFS server related to OpenBSD (if I understand correctly) virtualization. Are you trying to store OpenBSD vmm images on the NFS share exported from a DF server? That is a really, really bad idea. https://marc.info/?l=dragonfly-users=140384130921709=2 > > DragonFly which gets it software RAID discipline through old > > unmaintained FreeBSD natacontrol utility. Hardware RAID cards are not > > frequently tested and community seems to be keen on treating DF as a > > desktop OS rather than a storage workhorse. Having said that HDD are > > cheap this days and home users probably don't need anything bigger than > > a 12TB mirror. > I dont store much anyways, so I'll see as I go. > 12 TB is the sweet spot when it comes GB/dollar for platter HDDs. Predrag > Regards
Re: Home NAS
Pardon my ignorance and lack of deeper knowledge regarding the matter, but since when is XFS not even considered for such uses? Cheers Milun On Sun, Nov 17, 2019, 21:11 Patrick Marchand wrote: > Hello, > > On 11/15, Predrag Punosevac wrote: > > Patrick Marchand wrote: > > > I'll be playing around with DragonflyBSD Hammer2 (and multiple offsite > > > backups) for a home NAS over the next few weeks. I'll probably do a > > > presentation about the experience at the Montreal BSD user group > > > afterwards. It does not require as many ressources as ZFS or BTRFS, > > > but offers many similar features. > > > > > > > Been there, done that! > Cool ! I might ping you off-list with questions when I get to it. > > > H2 lacks built in backup mechanism. I was hoping that H2 will get some > > kind "hammer mirror-copy" of H1, or "zfs send/receive". My server is > > still on H1 and I really enjoy being able to continuously back it up. > > That's the only thing I am missing in H2. On the positive note H2 did > > get support for boot environment manager last year. > > > > https://github.com/newnix/dfbeadm > > > > Also DF jails are stuck in 2004 or something like that. I like their > > NFSv3. > I'm not planning on using jails much, instead I'll be using the > DFly NFS with OpenBSD to experiment with virtualization. > > > DragonFly which gets it software RAID discipline through old > > unmaintained FreeBSD natacontrol utility. Hardware RAID cards are not > > frequently tested and community seems to be keen on treating DF as a > > desktop OS rather than a storage workhorse. Having said that HDD are > > cheap this days and home users probably don't need anything bigger than > > a 12TB mirror. > I dont store much anyways, so I'll see as I go. > > Regards > >
Re: Home NAS
Hello, On 11/15, Predrag Punosevac wrote: > Patrick Marchand wrote: > > I'll be playing around with DragonflyBSD Hammer2 (and multiple offsite > > backups) for a home NAS over the next few weeks. I'll probably do a > > presentation about the experience at the Montreal BSD user group > > afterwards. It does not require as many ressources as ZFS or BTRFS, > > but offers many similar features. > > > > Been there, done that! Cool ! I might ping you off-list with questions when I get to it. > H2 lacks built in backup mechanism. I was hoping that H2 will get some > kind "hammer mirror-copy" of H1, or "zfs send/receive". My server is > still on H1 and I really enjoy being able to continuously back it up. > That's the only thing I am missing in H2. On the positive note H2 did > get support for boot environment manager last year. > > https://github.com/newnix/dfbeadm > > Also DF jails are stuck in 2004 or something like that. I like their > NFSv3. I'm not planning on using jails much, instead I'll be using the DFly NFS with OpenBSD to experiment with virtualization. > DragonFly which gets it software RAID discipline through old > unmaintained FreeBSD natacontrol utility. Hardware RAID cards are not > frequently tested and community seems to be keen on treating DF as a > desktop OS rather than a storage workhorse. Having said that HDD are > cheap this days and home users probably don't need anything bigger than > a 12TB mirror. I dont store much anyways, so I'll see as I go. Regards
Re: Home NAS
Hi, I found it, there exist glastree which is available from ports. Nice small "poor man's" backup as the author qualifies, though makes incremental backup through hard links: # if yesterday does not exist or today is newer, copy the file # else hard link the file to yesterday Ports: http://ports.su/sysutils/glastree Source: https://github.com/jeremywohl/glastree You can simply run it from crontab and even setup a short time daily and long time monthly or what ever else suits best through running the utility with different configurations from multiple crontab lines (daily, monthly, etc ...) glastree-1.04p0 – poor man's daily snapshot The poor man's daily snapshot, glastree builds live backup trees, with branches for each day. Users directly browse the past to recover older documents or retrieve lost files. Hard links serve to compress out unchanged files, while modified ones are copied verbatim. A prune utility effects a constant, sliding window. Satoru Takabayashi has written a similar program, in Ruby, pdumpfs. Inspired by Plan9, of course. Regards, Jean-Francois Le 15 nov. 2019 à 11:04, Raf Czlonka a écrit : > On Fri, Nov 15, 2019 at 08:54:54AM GMT, Andrew Luke Nesbit wrote: >> On 15/11/2019 10:11, gwes wrote: >> >>> The backup(8) program can assist this by storing deltas so that >>> more frequent backups only contain deltas from the previous >>> less frequent backup. >> >> I've not used backup(8) before, thanks for the suggestion. I will have a >> look. >> > > Hi Andrew, > > There is no backup(8) - gwes either meant a generic "backup" software, > or dump(8), and restore(8), specifically. > > Regards, > > Raf >
Re: Home Nas -> Montreal BUG
Hey, Since I'm getting off-list questions from more than one person, I'll post here as well. On 11/15, Patrick Marchand wrote: > I'll be playing around with DragonflyBSD Hammer2 (and multiple offsite > backups) for a home NAS over the next few weeks. I'll probably do a > presentation about the experience at the Montreal BSD user group afterwards. > It does not require as many ressources as ZFS or BTRFS, but offers many > similar features. It's brand spanking new and I havent advertised much beyond Mastodon and l'Agenda du Libre. (Somebody at work posted it in the work owned meetup, but that's not a platform I want as a main base). The actual link for the next event (on the 27th of november) is here: https://agendadulibre.qc.ca/events/2050, I'm planning it as a free (as in free of scheduled activities) session where people will have access to food, drinks and wifi so they can ask configuration questions, install a BSD, work on a port, etc.. We've had one meeting already (https://agendadulibre.qc.ca/events/2038), where I presented my experience with building an OpenBSD based home router and we had a round table of discussion on BSD systems in general. Including me and two work colleagues, we were 10 people (I honestly thought nobody would come at first, so I'm happy!). I'll post the slides online soon. Format wise I'm thinking of holding it monthly, alternating presentations and free sessions. Presentations can be given in the language of your choice, but french is encouraged. Nothing is set in stone yet, so we'll see how it evolves. I do need to create a website, mailing list and caldav for it, but for now questions and ideas for presentations can be sent to m...@patrickmarchand.com or @mathuin@bsd.network (mastodon) Au plaisir
Re: Home NAS
On 2019-11-15 14:34, Rafael Possamai wrote: My experience with ZFS (FreeNAS for the most part) is that it becomes more "expensive" to expand your pool after the fact (for a couple of different reasons, see below), That's probably case with more complex ZFS RAID setup, but for this particular usage RAID1 should be fine (OP preferred AFAIK on backup box) and expanding ZFS RAID1 is a matter of few commands and hardware work. I expanded ZFS RAID1 from old SXDE I don't remember year, 160 GB -> 500 GB -> 1TB over the years. I used the same trick to even switch from hdd to ssd, 1TB -> 1TB. Basically the only thing you need is to put your new drive(s) into the system and attach (zfs attach!) it to mirror, wait for resilver and then detach (zfs detach!) the old drives. You can then detach drives physically form the box. Anyway once your mirror is built of bigger drives it expands automatically. Karel
Re: Home NAS
On 2019-11-14 15:26, Jan Betlach wrote: Hi guys, I am setting up a home NAS for five users. Total amount of data stored on NAS will not exceed 5 TB. Clients are Macs and OpenBSD machines, so that SSHFS works fine from both (no need for NFS or Samba). I am much more familiar and comfortable with OpenBSD than with FreeBSD. My dilema while stating the above is as follows: Will the OpenBSD’s UFS stable and reliable enough for intended purpose? Software-wise, OpenBSD's UFS is probably rock solid. The only possible issues reported going years back is usage of softdep which is not switched on by default so you don't need to worry anyway. NAS will consist of just one encrypted drive, regularly backed to hardware RAID encrypted two-disks drive via rsync. I'm not sure this is the best strategy. Should I byte the bullet and build the NAS on FreeBSD taking advantage of ZFS, snapshots, replications, etc? Or is this an overkill? ZFS is never overkill. Be you, I would consider to use ZFS for NAS with 2 drives in RAID1 -- so self healing is usable if some drive or cable is not healthy enough. And then backup that to another ZFS box with again 2 drive in RAID1. If you would use kind of similar OSes there with the same ZFS level support, then you would be even able to use zfs send/receive functionality which will easy/speedup a lot your NAS backups to the backup server/NAS. Karel
Re: Home NAS
On 2019-11-15 16:02, pierre1.bar...@orange.com wrote: Hello, I tried a home NAS with ZFS, then BTRFS. Those filesystems needs tons of RAM (~1 GB of RAM by TB of disk), preferably ECC. For NAS you prefer ECC anyway and 1 GB RAM consumption per 1 TB of drive is urban legend probably passed by folks using deduplication. OP even does not list dedup requirements so I'd leave that out of table and reduce whatever ZFS RAM usage is by appropriate ZFS option. Years ago I started using ZFS on 1GB RAM/AMD Athlon 64 and it was fine, I just lowered ZFS cache RAM limit a bit. I found it very expensive for home usage, so I wouldn't recommend it. I'm running it on several workstations (SunOS/Linux) and most importantly (for OP) also on very old wife's computer (athlon x2, 4GB RAM -- only!) and it runs fine. Wife's computer is primary example since up to recently I've got several checksum errors on it per week (btw all computers with ZFS are running RAID1 mirror to support self-healing) which I fixed after several months by switching to new SATA cables. But anyway, ZFS saved quite a lot of data form errors which would go unnoticed if she would run anything else (well besides btrfs and my experimental RAID1 softraid with checksumming I'm tinkering on another box here. :-) Recovy systems were also inexistent at the time (no btrfsck), I don't know if it has improved since. I ended with LVM : cheap to implement and very easy to extend. I am very happy with it. And bitrot solution? Karel
Re: Home NAS
Hi, I remind there was an incremental backup which I used to run in cron, doing good job of making daily, weekly and monthly backups of deltas. I could not find the name of this, it was available from packages as far as I remember and created directory trees to the dates filled in with only modified files. Jean-François Le 15 nov. 2019 à 11:04, Raf Czlonka a écrit : > On Fri, Nov 15, 2019 at 08:54:54AM GMT, Andrew Luke Nesbit wrote: >> On 15/11/2019 10:11, gwes wrote: >> >>> The backup(8) program can assist this by storing deltas so that >>> more frequent backups only contain deltas from the previous >>> less frequent backup. >> >> I've not used backup(8) before, thanks for the suggestion. I will have a >> look. >> > > Hi Andrew, > > There is no backup(8) - gwes either meant a generic "backup" software, > or dump(8), and restore(8), specifically. > > Regards, > > Raf >
Re: Home NAS
Paolo Aglialoro wrote: > A fundamental element missing from the 1st mail is on which hardware should > run your software-defined NAS and for which use. > > I exclude you are talking about several nodes, on which you can run Ceph or > GlusterFS filesystems. > "Ceph & Gluster are WILDLY different solutions to different problems." https://www.reddit.com/r/sysadmin/comments/9onemk/ceph_vs_glusterfs/ OP is taking about home NAS. That pretty much means that the files will be accessed by SSHFS, NFS, or CIFS. Note that OmniOS has a kernel implementation of CIFS unlike FreeBSD. GlusterFS just like SSHFS, NFS, or CIFS allows access to files from multiple hosts sharing via a computer network with some added dough. Those files still have to be stored on the HW/Soft RAID on the top of some file system UFS, H1/2, ZFS, XFS. I am only aware of the native Linux GlusterFS client. For all practical purposes you will be using NFS client. Deciding between GlusterFS vs NFS is not an easy thing https://www.catalyst.net.nz/blog/our-glusterfs-experiences Quite a few US national laboratories and super computing centers use GlusterFS typically Red Hat (or clones) servers. Red Hat has officially support only XFS storage. However most people use ZFS as a backend. ZFS is a third party kernel module on Linux which is very adverse to such modules. It is major pain in the rear end to run comparing to Illumos or FreeBSD but I know lot of big shops who are doing exactly that. Personally if I had to design such a large network-attached distributed storage file system I would use non-Linux ZFS for a backend which will be mounted via iSCSI on Red Hat (or clone) GlusterFS servers. FreeBSD iSCSI implementation used to be PITA and an afterthought. Illumis has an excellent iSCSI implementation but I understand why most people will be apprehensive about deploying anything Illumos based. I have no idea how is any of this related to OpenBSD or for that matter any BSD since initial FreeBSD port of GlusterFS is obsolete not upgraded for many years. > Is it a single full size multi-disk server planned for intensive activity? > In this case don't reinvent the wheel, you got: > - FreeNAS > - napp-it (over solaris/omnios/openindiana) > - Nexenta This is a really bad advise! As somebody who foolishly built a few FreeNAS based sites and dismantled many more as paid jobs I could not agree more with this blog post https://smbitjournal.com/tag/freenas/ Again this has nothing to do with OpenBSD. OpenBSD file server with soft RAID1 mirror (for high availability and redundancy) will be more than adequate for most home users. It is super simple and sysutils/bitrot is sufficient protection from slow decay. Predrag > Just don't forget to substitute whatever raid SAS controller with an IT > mode enabled one (e.g. LSI 2308) in order to really benefit of ZFS. > > Is it for home use? Why not considering some low consumption hardware? If > you want multidisk RAID just buy a qnap/synology. > > If one disk is enough, buy Odroid HC2 which mounts 3.5" SATA disks, where a > 6TB one fits perfectly. Dunno if OpenBSD may install on it (armhf v7 arch), > but for sure either armbian or openmediavault are good choices to run on, > having full 1Gb/s throughput and consuming even less current than some > famous brand NAS, like the ones named before. > > This said, if the aim of the project is just having fun creating a NAS from > scratch on casual hardware running OpenBSD for the sake of it, I shut my > mouth. > > Have phun!
Re: Home NAS
A fundamental element missing from the 1st mail is on which hardware should run your software-defined NAS and for which use. I exclude you are talking about several nodes, on which you can run Ceph or GlusterFS filesystems. Is it a single full size multi-disk server planned for intensive activity? In this case don't reinvent the wheel, you got: - FreeNAS - napp-it (over solaris/omnios/openindiana) - Nexenta Just don't forget to substitute whatever raid SAS controller with an IT mode enabled one (e.g. LSI 2308) in order to really benefit of ZFS. Is it for home use? Why not considering some low consumption hardware? If you want multidisk RAID just buy a qnap/synology. If one disk is enough, buy Odroid HC2 which mounts 3.5" SATA disks, where a 6TB one fits perfectly. Dunno if OpenBSD may install on it (armhf v7 arch), but for sure either armbian or openmediavault are good choices to run on, having full 1Gb/s throughput and consuming even less current than some famous brand NAS, like the ones named before. This said, if the aim of the project is just having fun creating a NAS from scratch on casual hardware running OpenBSD for the sake of it, I shut my mouth. Have phun! Il sab 16 nov 2019, 07:11 Jordan Geoghegan ha scritto: > > On 2019-11-15 20:47, Predrag Punosevac wrote: > > Jan Betlach wrote: > > > > [snip] > > > >> 2. A HP P222 array controller works right out of the box on > >> OpenBSD, maybe FreeBSD as well but the combination of ZFS and RAID > >> controller seems weird to me. > >> > > FreeBSD has a better support for HWRaid cards than OpenBSD. I am talking > > about serious HWRaid cards like former LSI Controllers. Only Areca used > > to fully support OpenBSD. Also FreeBSD UFS journaling is more advanced > > than OpenBSD journaling. > > OpenBSD's UFS doesn't do any journalling. > > [snip] > > >> 3. OpenBSD is actually out of my expectation. CIFS and NFS is just > >> easy to setup. The most fabulous thing to me is the full disk > >> encryption. I had a disk failure and the array controller was burnt > >> once because I had some cooling issue. However, I was confident to get > >> a replacement and no data was lost. > > > > OpenBSD NFS server implementation is slow comparing to others but for > > home users YMMV. > I was able to get Gigabit line rate from an OpenBSD NAS to CentOS > clients no problem. The OpenBSD NFS client is admittedly somewhat slow-- > I was only able to get ~70MB/s out of it when connected to the same NAS > that gets 100MBps+ from Linux based NFS clients. > > > > Code: > > # bioctl sd4 > > Volume Status Size Device > > softraid0 0 Online 2000396018176 sd4 RAID1 > >0 Online 2000396018176 0:0.0 noencl > >1 Online 2000396018176 0:1.0 noencl > > > > is very crude. It took me 4 days to rebuild 1TB mirror after accidental > > power off one HDD. That is just not something usable for a storage > > purpose in real life. > > I have an OpenBSD NAS at home with 20TB of RAID1 storage comprised of 10 > 4TB drives. Last time I had to rebuild one of the arrays, it took just > under 24 hours to rebuild. This was some months ago, but I remember > doing the math and I was getting just under 50MB/s rebuild speed. This > was on a fairly ancient Xeon rig using WD Red NAS drives. If it took > your machine 4 days to rebuild a 1TB mirror, something must be wrong, > possibly hardware related as that's less than 4MB/s rebuild speed. > > > > > At work where I have to store petabytes of data I use only ZFS. At home > > that is another story. > > > > For the record BTRFS is a vaporware and I would never store the pictures > > of my kids to that crap. > > > > Cheers, > > Predrag > > Cheers, > > Jordan > >
Re: Home NAS
On 2019-11-15 20:47, Predrag Punosevac wrote: Jan Betlach wrote: [snip] 2. A HP P222 array controller works right out of the box on OpenBSD, maybe FreeBSD as well but the combination of ZFS and RAID controller seems weird to me. FreeBSD has a better support for HWRaid cards than OpenBSD. I am talking about serious HWRaid cards like former LSI Controllers. Only Areca used to fully support OpenBSD. Also FreeBSD UFS journaling is more advanced than OpenBSD journaling. OpenBSD's UFS doesn't do any journalling. [snip] 3. OpenBSD is actually out of my expectation. CIFS and NFS is just easy to setup. The most fabulous thing to me is the full disk encryption. I had a disk failure and the array controller was burnt once because I had some cooling issue. However, I was confident to get a replacement and no data was lost. OpenBSD NFS server implementation is slow comparing to others but for home users YMMV. I was able to get Gigabit line rate from an OpenBSD NAS to CentOS clients no problem. The OpenBSD NFS client is admittedly somewhat slow-- I was only able to get ~70MB/s out of it when connected to the same NAS that gets 100MBps+ from Linux based NFS clients. Code: # bioctl sd4 Volume Status Size Device softraid0 0 Online 2000396018176 sd4 RAID1 0 Online 2000396018176 0:0.0 noencl 1 Online 2000396018176 0:1.0 noencl is very crude. It took me 4 days to rebuild 1TB mirror after accidental power off one HDD. That is just not something usable for a storage purpose in real life. I have an OpenBSD NAS at home with 20TB of RAID1 storage comprised of 10 4TB drives. Last time I had to rebuild one of the arrays, it took just under 24 hours to rebuild. This was some months ago, but I remember doing the math and I was getting just under 50MB/s rebuild speed. This was on a fairly ancient Xeon rig using WD Red NAS drives. If it took your machine 4 days to rebuild a 1TB mirror, something must be wrong, possibly hardware related as that's less than 4MB/s rebuild speed. At work where I have to store petabytes of data I use only ZFS. At home that is another story. For the record BTRFS is a vaporware and I would never store the pictures of my kids to that crap. Cheers, Predrag Cheers, Jordan
Re: Home NAS
Jan Betlach wrote: > - FFS seems to be reliable and stable enough for my purpose. ZFS is too > complicated and bloated (of course it has its advantages), however major > factor for me has been that it is not possible to encrypt ZFS natively > on FreeBSD as of now. Illumos distro OmniOS CE https://omniosce.org/ has support for native encryption since r151032 https://github.com/omniosorg/omnios-build/blob/r151032/doc/ReleaseNotes.md Patrick Marchand wrote: > Hi, > > > I'll be playing around with DragonflyBSD Hammer2 (and multiple offsite > backups) for a home NAS over the next few weeks. I'll probably do a > presentation about the experience at the Montreal BSD user group > afterwards. It does not require as many ressources as ZFS or BTRFS, > but offers many similar features. > Been there, done that! dfly# uname -a DragonFly dfly.int.bagdala2.net 5.6-RELEASE DragonFly v5.6.2-RELEASE #26: Sun Aug 11 16:04:07 EDT 2019 r...@dfly.int.bagdala2.net:/usr/obj/usr/src/sys/X86_64_GENERIC x86_64 # DeviceMountpoint FStype Options Dump Pass# /dev/serno/B620550018.s1a /boot ufs rw 1 1 # /dev/serno/B620550018.s1b noneswapsw 0 0 # Next line adds swapcache on the separate HDD instead of original swap commented out above /dev/serno/451762B0E46228230099.s1b noneswap sw 0 0 /dev/serno/B620550018.s1d / hammer rw 1 1 /pfs/var/varnullrw 0 0 /pfs/tmp/tmpnullrw 0 0 /pfs/home /home nullrw 0 0 /pfs/usr.obj/usr/objnullrw 0 0 /pfs/var.crash /var/crash nullrw 0 0 /pfs/var.tmp/var/tmpnullrw 0 0 proc/proc procfs rw 0 0 # Added by Predrag Punosevac /dev/serno/ZDS01176.s1a /data hammer rw 2 2 /dev/serno/5QG00WTH.s1a /mirror hammer rw 2 2 # /dev/serno/5QG00XF0.s1e /test-hammer2 hammer2 rw 2 2 # Mount pseudo file systems from the master drive which is used as a backup for my desktop /data/pfs/backups /data/backups nullrw 0 0 /data/pfs/nfs /data/nfs nullrw 0 0 H2 lacks built in backup mechanism. I was hoping that H2 will get some kind "hammer mirror-copy" of H1, or "zfs send/receive". My server is still on H1 and I really enjoy being able to continuously back it up. That's the only thing I am missing in H2. On the positive note H2 did get support for boot environment manager last year. https://github.com/newnix/dfbeadm Also DF jails are stuck in 2004 or something like that. I like their NFSv3. DragonFly which gets it software RAID discipline through old unmaintained FreeBSD natacontrol utility. Hardware RAID cards are not frequently tested and community seems to be keen on treating DF as a desktop OS rather than a storage workhorse. Having said that HDD are cheap this days and home users probably don't need anything bigger than a 12TB mirror. Zhi-Qiang Lei wrote: > 1. FreeBSD was my first consideration because of ZFS, but as far as I > know, ZFS doesn't work well with RAID controller, Of course not. ZFS is a volume manager and file system in one. How would ZFS detect errors and do self-healing if it relies on the HW Raid controller to get the info about block devices? > and neither FreeBSD > nor OpenBSD has a driver for the B120i array controller on the > mainboard (HP is to be blamed). I could use AHCI mode instead RAID > which also suits ZFS of FreeBSD, yet there is a notorious fan noise > issue of that approach. > That is not a genuine HWRaid card. That is a build in software raid. You should not be using that crap. > 2. A HP P222 array controller works right out of the box on > OpenBSD, maybe FreeBSD as well but the combination of ZFS and RAID > controller seems weird to me. > FreeBSD has a better support for HWRaid cards than OpenBSD. I am talking about serious HWRaid cards like former LSI Controllers. Only Areca used to fully support OpenBSD. Also FreeBSD UFS journaling is more advanced than OpenBSD journaling. However unless you put H1 on H2 on the top of hardware RAID you will not get COW, snapshots, history, and all other stuff with any version of UFS. I know people on this list who prefer HWRaid and also know people on this list who prefer softward (including ZFS). > 3. OpenBSD is actually out of my expectation. CIFS and NFS is just > easy to setup. The most fabulous thing to me is the full disk > encryption. I had a disk failure and the array controller was burnt > once because I had some cooling issue. However, I was confident to get > a replacement and no data was lost. OpenBSD NFS server
Re: Home NAS
[misc intermediate comments removed] On 11/15/19 3:54 AM, Andrew Luke Nesbit wrote: In particular I'm trying to figure out a generally applicable way of taking a _consistent_ backup of a disk without resorting to single user mode. I think COW file systems might help in this regard but I don't think anything like this exists in OpenBSD. COW in the filesystem, no. However... a backup is a precautionary copy-before-write. The only difference is the time granularity. Consistency? An arbitrary file system snapshot doesn't guarantee that you won't see -application level- inconsistent data, just that the files didn't change during backup. Even a COW system that doesn't reveal a new version of a file until it's been closed won't protect you from an inconsistent group of files. What groups of files --must-- be perfectly archived? If you (a) can pause user activity (b) can tolerate some inconsistency in captured system log files, then just run a backup. Partial DB transactions had better be OK or get a new DBM. You might have to pause cron(8). I don't remember any other daemon that would care. Some archive programs will complain if the size or modification time of a file changes during copy or if the directory contents change. Something could be done to automatically go back for those. Depending very much on your mix of uses, don't even stop anything. Breaking up the backup into sections - per project, per file system, etc. can make the pauses less objectionable. It can make recovery easier as well. Assuming you have control over the system files those only need a couple of copies when they change, for instance. Brute force: ls -Rl /users /proj1 /proj2 > before0 $BACKUP -o /$BACKUPFS/backup-$(date) ls -Rl /users /proj1 /proj2 > after0 # remove known don't-cares sed -f ignores before0 > before sed -f ignores after0 > after # check to see if any action needed diff before after > changed grep -f vitalfiles changed > urgent cat urgent changed | mail -s "changes during backup $(date)" you # calculate list of files needing recopy $SCRIPT changed > newbackup # copy files missed - should run quickly $BACKUP -o /$BACKUPFS/bdelta-$(date) -f newbackup This worked pretty well for me. The truly paranoid would put a while loop around the diff & recopy... Binary files can be regenerated if the source *and* environment are backed up. Storing the environment is a tricky problem that I haven't found an entirely satisfactory solution for, yet. The key is for the project never to use an unqualified program - always "our current version". One solution is to copy or link a consistent set of utilities (compiler, linker, libraries) into the project and always use those in production. Then a backup will capture everything. This won't necessarily work if the OS changes its ABI but it can be pretty effective. I've been in a project that used this approach and it did work. Keeping an automatic record of utility and library versions used works as long as the system itself is backed up well. The discipline to keep everything tidy, ... well. Without regard to backups, the precaution to take periodic snapshots of a project, transplant it into an empty system and make sure the snapshot actually works has been erm, revealing. # mv /usr/bin/cc /usr/bin/saved-cc # rm /usr/bin/cc $ make .not found Andrew It can be a pain to design a procedure that fits your needs and doesn't need a staff of operators (:-( Good luck! Geoff Steckel
Re: Home NAS
I have a HP Gen8 Microserver running as a NAS using OpenBSD. It has been serving well for like 5 months. I choose OpenBSD over FreeBSD because: 1. FreeBSD was my first consideration because of ZFS, but as far as I know, ZFS doesn’t work well with RAID controller, and neither FreeBSD nor OpenBSD has a driver for the B120i array controller on the mainboard (HP is to be blamed). I could use AHCI mode instead RAID which also suits ZFS of FreeBSD, yet there is a notorious fan noise issue of that approach. 2. A HP P222 array controller works right out of the box on OpenBSD, maybe FreeBSD as well but the combination of ZFS and RAID controller seems weird to me. 3. OpenBSD is actually out of my expectation. CIFS and NFS is just easy to setup. The most fabulous thing to me is the full disk encryption. I had a disk failure and the array controller was burnt once because I had some cooling issue. However, I was confident to get a replacement and no data was lost. As the 5TB limitation, I haven’t been there. > On Nov 14, 2019, at 10:26 PM, Jan Betlach wrote: > > > Hi guys, > > I am setting up a home NAS for five users. Total amount of data stored on NAS > will not exceed 5 TB. > Clients are Macs and OpenBSD machines, so that SSHFS works fine from both (no > need for NFS or Samba). > I am much more familiar and comfortable with OpenBSD than with FreeBSD. > My dilema while stating the above is as follows: > > Will the OpenBSD’s UFS stable and reliable enough for intended purpose? NAS > will consist of just one encrypted drive, regularly backed to hardware RAID > encrypted two-disks drive via rsync. > > Should I byte the bullet and build the NAS on FreeBSD taking advantage of > ZFS, snapshots, replications, etc? Or is this an overkill? > > BTW my most important data is also backed off-site. > > Thank you in advance for your comments. > > Jan >
Re: Home NAS
Hi, I'll be playing around with DragonflyBSD Hammer2 (and multiple offsite backups) for a home NAS over the next few weeks. I'll probably do a presentation about the experience at the Montreal BSD user group afterwards. It does not require as many ressources as ZFS or BTRFS, but offers many similar features. As for OpenBSD as a home NAS, I'm sure it would be fine if you're diligent about backups. You might also want to get a small UPS for it (I bought a refurbished APC for 80$), so frequent but short power interruptions do not require an FFS integrity check. (I do this for my router) Pleasure
Re: Home NAS
Hi, thank you all for comments. I am restoring backup to my new OpenBSD based home NAS as of writing this. Why I have decided to go this route and not with other option like ZFS: - FFS seems to be reliable and stable enough for my purpose. ZFS is too complicated and bloated (of course it has its advantages), however major factor for me has been that it is not possible to encrypt ZFS natively on FreeBSD as of now. I am also more comfortable with Open BSD than with Free BSD. I did not want to go with Linux at all. - I have installed Open BSD on an external unencrypted USB stick. So that I don’t need to have access to the box in case of restart. Main data NAS disk is 2TB internal one in the box (Zotac nano), which is encrypted. I can easily mount it via SSH in case of restart. Backups are automated via rsync to the encrypted external hardware RAID disks. Using DUIDs for all drives. - I do keep offsite backup as well. I have tested this setup in the last couple of days before going all in. So far so good. Performance is plenty acceptable for my usage. Mounting the NAS storage via SSHFS on client machines (Macs and OpenBSDs) works flawlessly and speed is also OK. Thanks again Jan On 15 Nov 2019, at 16:02, pierre1.bar...@orange.com wrote: Hello, I tried a home NAS with ZFS, then BTRFS. Those filesystems needs tons of RAM (~1 GB of RAM by TB of disk), preferably ECC. I found it very expensive for home usage, so I wouldn't recommend it. Recovy systems were also inexistent at the time (no btrfsck), I don't know if it has improved since. I ended with LVM : cheap to implement and very easy to extend. I am very happy with it. -- Cordialement, Pierre BARDOU -Message d'origine- De : owner-m...@openbsd.org De la part de Rafael Possamai Envoyé : vendredi 15 novembre 2019 14:35 À : Jan Betlach Cc : misc@openbsd.org Objet : Re: Home NAS My experience with ZFS (FreeNAS for the most part) is that it becomes more "expensive" to expand your pool after the fact (for a couple of different reasons, see below), but if 5TB is all you're ever going to need in this specific case, I think you should be fine and can take advantage of ZFS features like you said. I have sources for this at home (a couple of articles and link to a forum thread), but these are saved on my desktop at home. Just let me know and I'll share them with you later. On Thu, Nov 14, 2019, 8:27 AM Jan Betlach wrote: Hi guys, I am setting up a home NAS for five users. Total amount of data stored on NAS will not exceed 5 TB. Clients are Macs and OpenBSD machines, so that SSHFS works fine from both (no need for NFS or Samba). I am much more familiar and comfortable with OpenBSD than with FreeBSD. My dilema while stating the above is as follows: Will the OpenBSD’s UFS stable and reliable enough for intended purpose? NAS will consist of just one encrypted drive, regularly backed to hardware RAID encrypted two-disks drive via rsync. Should I byte the bullet and build the NAS on FreeBSD taking advantage of ZFS, snapshots, replications, etc? Or is this an overkill? BTW my most important data is also backed off-site. Thank you in advance for your comments. Jan _ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you.
Re: Home NAS
Hello, I tried a home NAS with ZFS, then BTRFS. Those filesystems needs tons of RAM (~1 GB of RAM by TB of disk), preferably ECC. I found it very expensive for home usage, so I wouldn't recommend it. Recovy systems were also inexistent at the time (no btrfsck), I don't know if it has improved since. I ended with LVM : cheap to implement and very easy to extend. I am very happy with it. -- Cordialement, Pierre BARDOU -Message d'origine- De : owner-m...@openbsd.org De la part de Rafael Possamai Envoyé : vendredi 15 novembre 2019 14:35 À : Jan Betlach Cc : misc@openbsd.org Objet : Re: Home NAS My experience with ZFS (FreeNAS for the most part) is that it becomes more "expensive" to expand your pool after the fact (for a couple of different reasons, see below), but if 5TB is all you're ever going to need in this specific case, I think you should be fine and can take advantage of ZFS features like you said. I have sources for this at home (a couple of articles and link to a forum thread), but these are saved on my desktop at home. Just let me know and I'll share them with you later. On Thu, Nov 14, 2019, 8:27 AM Jan Betlach wrote: > > Hi guys, > > I am setting up a home NAS for five users. Total amount of data stored > on NAS will not exceed 5 TB. > Clients are Macs and OpenBSD machines, so that SSHFS works fine from > both (no need for NFS or Samba). > I am much more familiar and comfortable with OpenBSD than with FreeBSD. > My dilema while stating the above is as follows: > > Will the OpenBSD’s UFS stable and reliable enough for intended > purpose? NAS will consist of just one encrypted drive, regularly > backed to hardware RAID encrypted two-disks drive via rsync. > > Should I byte the bullet and build the NAS on FreeBSD taking advantage > of ZFS, snapshots, replications, etc? Or is this an overkill? > > BTW my most important data is also backed off-site. > > Thank you in advance for your comments. > > Jan > > _ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you.
Re: Home NAS
My experience with ZFS (FreeNAS for the most part) is that it becomes more "expensive" to expand your pool after the fact (for a couple of different reasons, see below), but if 5TB is all you're ever going to need in this specific case, I think you should be fine and can take advantage of ZFS features like you said. I have sources for this at home (a couple of articles and link to a forum thread), but these are saved on my desktop at home. Just let me know and I'll share them with you later. On Thu, Nov 14, 2019, 8:27 AM Jan Betlach wrote: > > Hi guys, > > I am setting up a home NAS for five users. Total amount of data stored > on NAS will not exceed 5 TB. > Clients are Macs and OpenBSD machines, so that SSHFS works fine from > both (no need for NFS or Samba). > I am much more familiar and comfortable with OpenBSD than with FreeBSD. > My dilema while stating the above is as follows: > > Will the OpenBSD’s UFS stable and reliable enough for intended > purpose? NAS will consist of just one encrypted drive, regularly backed > to hardware RAID encrypted two-disks drive via rsync. > > Should I byte the bullet and build the NAS on FreeBSD taking advantage > of ZFS, snapshots, replications, etc? Or is this an overkill? > > BTW my most important data is also backed off-site. > > Thank you in advance for your comments. > > Jan > >
Re: Home NAS
I don't know how current tape systems are, but I have been burnt by them in the past. Either the tape deteriorates or the tape writer company goes out of business. My current approach is to keep stuff I want to keep on current online storage in multiple places plus offline USB. Data get migrated to new media as they appear and prove themselves. There is still the possibility of undetected bit rot however... Dave On 11/15/19, Andrew Luke Nesbit wrote: > On 15/11/2019 10:11, gwes wrote: >> On 11/14/19 3:52 PM, Andrew Luke Nesbit wrote: >>> On 15/11/2019 07:44, Raymond, David wrote: I hadn't heard about file corruption on OpenBSD. It would be good to get to the bottom of this if it occurred. >>> >>> I was surprised when I read mention of it too, without any real claim >>> or detailed analysis to back it up. This is why I added my disclaimer >>> about "correcting me if I'm wrong because I don't want to spread >>> incorrect information". > > [...] > >> There was a thread a couple of months ago started by someone either >> pretty >> ignorant or a troll. >> The consensus answer: no more than any other OS, less than many. > > Thank you gwes, for the clarification. > > The thread is vaguely coming back to my memory now. I was dipping in > and out of it at the time as I didn't have time to study the details at > the time. > >> One size definitely doesn't fit all. > > That is pretty obvious. I never mentioned a blanket rule, and I assume > that OP is able to tailor any suggestion to their needs. > >> Backup strategies depend on user's criteria, cost of design and >> cost of doing the backups - administration & storage, etc. > > Sure. I don't have a personal archival storage system yet for long term > storage that satisfies my specifications because I don't have the > infrastructure and medium yet to store it. I plan on investing in LTO > tape but I can not afford the initial cost yet. > >> In an ideal world every version of every file lasts forever. >> Given real limitations, versioning filesystems can't and don't. > > Indeed. But having archival snapshots at various points in time > increases the _probability_ that the version of the file that you need > will be present if+when you need it. > >> If your data are critical, invest in a dozen or more portable >> USB drives. Cycle them off-site. Reread them (not too often) >> to check for decay. > > Yes, this is part of the backup system that I'm designing for my NAS, > but it's not so much for archiving. > >> If you have much available, get a >> modern tape system. > > Yes, as I mentioned above LTO would be great if+when I can afford it. > >> The backup system used over 50 years ago still suitable for many >> circumstances looks something like this: >>daily backups held for 1 month >>weekly backups held for 6-12 months >>monthly backups held indefinitely offsite. >> Hold times vary according to circumstances. > > I think something like this is a good plan. > >> The backup(8) program can assist this by storing deltas so that >> more frequent backups only contain deltas from the previous >> less frequent backup. > > I've not used backup(8) before, thanks for the suggestion. I will have > a look. > >> The compromise between backup storage requirements and granularity >> of recovery points can be mitigated. The way to do it depends on >> the type and structure of the data: >> >> Some data are really transient and can be left out. >> >> Source code control systems (or whatever the name is this week) >> are a good way for intermittent backups to capture a good history >> of whatever data is around if it's text. > > I don't understand how SCM's are supposed to help with this... > >> DBs often have their own built-in backup mechanisms. > > This underscores the difference between file system-level backups, > block-level backups, and (for DBs) application-level backups. In > particular I'm trying to figure out a generally applicable way of taking > a _consistent_ backup of a disk without resorting to single user mode. > > I think COW file systems might help in this regard but I don't think > anything like this exists in OpenBSD. > >> Binary files can be regenerated if the source *and* environment >> are backed up. > > Storing the environment is a tricky problem that I haven't found an > entirely satisfactory solution for, yet. > >> been there, mounted the wrong tape... what write protect ring? > > O yeah... me too. My team inherited a hosted service and upon > auditing we discovered its backup system was stranger than fiction. But > it was so bizarre that we couldn't determine whether it was _supposed_ > to be that way or if our reasoning was flawed. A classic type of problem. > > Andrew > -- > OpenPGP key: EB28 0338 28B7 19DA DAB0 B193 D21D 996E 883B E5B9 > -- David J. Raymond david.raym...@nmt.edu http://physics.nmt.edu/~raymond
Re: Home NAS
On Fri, Nov 15, 2019 at 08:54:54AM GMT, Andrew Luke Nesbit wrote: > On 15/11/2019 10:11, gwes wrote: > > > The backup(8) program can assist this by storing deltas so that > > more frequent backups only contain deltas from the previous > > less frequent backup. > > I've not used backup(8) before, thanks for the suggestion. I will have a > look. > Hi Andrew, There is no backup(8) - gwes either meant a generic "backup" software, or dump(8), and restore(8), specifically. Regards, Raf
Re: Home NAS
On 15/11/2019 10:11, gwes wrote: On 11/14/19 3:52 PM, Andrew Luke Nesbit wrote: On 15/11/2019 07:44, Raymond, David wrote: I hadn't heard about file corruption on OpenBSD. It would be good to get to the bottom of this if it occurred. I was surprised when I read mention of it too, without any real claim or detailed analysis to back it up. This is why I added my disclaimer about "correcting me if I'm wrong because I don't want to spread incorrect information". [...] There was a thread a couple of months ago started by someone either pretty ignorant or a troll. The consensus answer: no more than any other OS, less than many. Thank you gwes, for the clarification. The thread is vaguely coming back to my memory now. I was dipping in and out of it at the time as I didn't have time to study the details at the time. One size definitely doesn't fit all. That is pretty obvious. I never mentioned a blanket rule, and I assume that OP is able to tailor any suggestion to their needs. Backup strategies depend on user's criteria, cost of design and cost of doing the backups - administration & storage, etc. Sure. I don't have a personal archival storage system yet for long term storage that satisfies my specifications because I don't have the infrastructure and medium yet to store it. I plan on investing in LTO tape but I can not afford the initial cost yet. In an ideal world every version of every file lasts forever. Given real limitations, versioning filesystems can't and don't. Indeed. But having archival snapshots at various points in time increases the _probability_ that the version of the file that you need will be present if+when you need it. If your data are critical, invest in a dozen or more portable USB drives. Cycle them off-site. Reread them (not too often) to check for decay. Yes, this is part of the backup system that I'm designing for my NAS, but it's not so much for archiving. If you have much available, get a modern tape system. Yes, as I mentioned above LTO would be great if+when I can afford it. The backup system used over 50 years ago still suitable for many circumstances looks something like this: daily backups held for 1 month weekly backups held for 6-12 months monthly backups held indefinitely offsite. Hold times vary according to circumstances. I think something like this is a good plan. The backup(8) program can assist this by storing deltas so that more frequent backups only contain deltas from the previous less frequent backup. I've not used backup(8) before, thanks for the suggestion. I will have a look. The compromise between backup storage requirements and granularity of recovery points can be mitigated. The way to do it depends on the type and structure of the data: Some data are really transient and can be left out. Source code control systems (or whatever the name is this week) are a good way for intermittent backups to capture a good history of whatever data is around if it's text. I don't understand how SCM's are supposed to help with this... DBs often have their own built-in backup mechanisms. This underscores the difference between file system-level backups, block-level backups, and (for DBs) application-level backups. In particular I'm trying to figure out a generally applicable way of taking a _consistent_ backup of a disk without resorting to single user mode. I think COW file systems might help in this regard but I don't think anything like this exists in OpenBSD. Binary files can be regenerated if the source *and* environment are backed up. Storing the environment is a tricky problem that I haven't found an entirely satisfactory solution for, yet. been there, mounted the wrong tape... what write protect ring? O yeah... me too. My team inherited a hosted service and upon auditing we discovered its backup system was stranger than fiction. But it was so bizarre that we couldn't determine whether it was _supposed_ to be that way or if our reasoning was flawed. A classic type of problem. Andrew -- OpenPGP key: EB28 0338 28B7 19DA DAB0 B193 D21D 996E 883B E5B9
Re: Home NAS
On 11/14/19 3:52 PM, Andrew Luke Nesbit wrote: Hi Dave, On 15/11/2019 07:44, Raymond, David wrote: I hadn't heard about file corruption on OpenBSD. It would be good to get to the bottom of this if it occurred. I was surprised when I read mention of it too, without any real claim or detailed analysis to back it up. This is why I added my disclaimer about "correcting me if I'm wrong because I don't want to spread incorrect information". The reason why I brought it up on a public mailing list was to find out if anybody else has heard any inkling _at all_ regarding this, even a skerrick of a mention. I have a feeling I may have even heard about it on this list but I'm not sure. If somebody out there genuinely suspects that this happened then it would be good to know so we can clear it up. Kind regards, Andrew There was a thread a couple of months ago started by someone either pretty ignorant or a troll. The consensus answer: no more than any other OS, less than many. On 11/14/19, U'll Be King of the Stars wrote: A couple of months ago I read a couple of reports of filesystem corruption on OpenBSD. I didn't have time to investigate deeply and I don't know if these issues were even real. Even if they were real I don't know if the problem was due to user error or a defect in the OS. This is an excellent reason for implementing a system that includes not only backups, but long term storage /archives/ too. Andrew One size definitely doesn't fit all. Backup strategies depend on user's criteria, cost of design and cost of doing the backups - administration & storage, etc. In an ideal world every version of every file lasts forever. Given real limitations, versioning filesystems can't and don't. If your data are critical, invest in a dozen or more portable USB drives. Cycle them off-site. Reread them (not too often) to check for decay. If you have much available, get a modern tape system. The backup system used over 50 years ago still suitable for many circumstances looks something like this: daily backups held for 1 month weekly backups held for 6-12 months monthly backups held indefinitely offsite. Hold times vary according to circumstances. The backup(8) program can assist this by storing deltas so that more frequent backups only contain deltas from the previous less frequent backup. The compromise between backup storage requirements and granularity of recovery points can be mitigated. The way to do it depends on the type and structure of the data: Some data are really transient and can be left out. Source code control systems (or whatever the name is this week) are a good way for intermittent backups to capture a good history of whatever data is around if it's text. DBs often have their own built-in backup mechanisms. Binary files can be regenerated if the source *and* environment are backed up. Etc. YMMV and MEGO geoff steckel been there, mounted the wrong tape... what write protect ring?
Re: Home NAS
I'm running a small home NAS on OpenBSD, in a very similar configuration as your intended configuration, right down to the rsync backup scripts. It's worked very well so far, though I've only had it in place for a bit over a year. I chose OpenBSD over FreeBSD due to being far more comfortable with OpenBSD than FreeBSD, and my desire to run my favorite OS in a meaningful way. I say go for it, and (like all systems) make sure your backups are good (that means test them periodically). -- Jon Tabor tab...@obsolete.site "I desire peace. I also *require* freedom. Not only for myself, but for Earth. If it can be achieved through peaceful ends, wonderful. War is waste. However, the only thing worse than war is the loss of liberty." -John Ringo, _The Hot Gate_
Re: Home NAS
Hi Dave, On 15/11/2019 07:44, Raymond, David wrote: I hadn't heard about file corruption on OpenBSD. It would be good to get to the bottom of this if it occurred. I was surprised when I read mention of it too, without any real claim or detailed analysis to back it up. This is why I added my disclaimer about "correcting me if I'm wrong because I don't want to spread incorrect information". The reason why I brought it up on a public mailing list was to find out if anybody else has heard any inkling _at all_ regarding this, even a skerrick of a mention. I have a feeling I may have even heard about it on this list but I'm not sure. If somebody out there genuinely suspects that this happened then it would be good to know so we can clear it up. Kind regards, Andrew On 11/14/19, U'll Be King of the Stars wrote: On 15/11/2019 04:45, Raymond, David wrote: I have done similar things on Linux for years and am now doing them on OpenBSD. Sounds like what you want to do can be done with a simple rsync script. OpenBSD ffs (ufs) should be stable, it has been around for decades in various incarnations. I have never noticed bit rot in this system, though I imagine it could happen if a disk is gradually going bad. Please correct me if I'm wrong because I don't want to spread incorrect information. A couple of months ago I read a couple of reports of filesystem corruption on OpenBSD. I didn't have time to investigate deeply and I don't know if these issues were even real. Even if they were real I don't know if the problem was due to user error or a defect in the OS. Does anybody know anything about this? That's why multiple backups help. Agreed. See below. You might want to set up a raid5 backup, as this detects parity errors. More complicated though. This is exactly the kind of reason that hybrid volume management systems + filesystems such as Btrfs and ZFS have become popular. I do not know anything about OpenBSD's LVM. One weakness in such as system (ask me how I know!) is that if the NAS goes gradually bad, the errors will propagate to the backup. Using rsync without the --delete option most of the time alleviates this somewhat. Only run with --delete when the backup starts getting full and you are confident that your NAS drive is ok. This is an excellent reason for implementing a system that includes not only backups, but long term storage /archives/ too. Andrew -- OpenPGP key: EB28 0338 28B7 19DA DAB0 B193 D21D 996E 883B E5B9 -- OpenPGP key: EB28 0338 28B7 19DA DAB0 B193 D21D 996E 883B E5B9
Re: Home NAS
Andrew, I hadn't heard about file corruption on OpenBSD. It would be good to get to the bottom of this if it occurred. Dave On 11/14/19, U'll Be King of the Stars wrote: > On 15/11/2019 04:45, Raymond, David wrote: >> I have done similar things on Linux for years and am now doing them on >> OpenBSD. Sounds like what you want to do can be done with a simple >> rsync script. OpenBSD ffs (ufs) should be stable, it has been around >> for decades in various incarnations. I have never noticed bit rot in >> this system, though I imagine it could happen if a disk is gradually >> going bad. > > Please correct me if I'm wrong because I don't want to spread incorrect > information. > > A couple of months ago I read a couple of reports of filesystem > corruption on OpenBSD. I didn't have time to investigate deeply and I > don't know if these issues were even real. Even if they were real I > don't know if the problem was due to user error or a defect in the OS. > > Does anybody know anything about this? > >> That's why multiple backups help. > > Agreed. See below. > >> You might want to set >> up a raid5 backup, as this detects parity errors. More complicated >> though. > > This is exactly the kind of reason that hybrid volume management systems > + filesystems such as Btrfs and ZFS have become popular. > > I do not know anything about OpenBSD's LVM. > >> One weakness in such as system (ask me how I know!) is that >> if the NAS goes gradually bad, the errors will propagate to the >> backup. Using rsync without the --delete option most of the time >> alleviates this somewhat. Only run with --delete when the backup >> starts getting full and you are confident that your NAS drive is ok. > > This is an excellent reason for implementing a system that includes not > only backups, but long term storage /archives/ too. > > Andrew > -- > OpenPGP key: EB28 0338 28B7 19DA DAB0 B193 D21D 996E 883B E5B9 > -- David J. Raymond david.raym...@nmt.edu http://physics.nmt.edu/~raymond
Re: Home NAS
On 15/11/2019 04:45, Raymond, David wrote: I have done similar things on Linux for years and am now doing them on OpenBSD. Sounds like what you want to do can be done with a simple rsync script. OpenBSD ffs (ufs) should be stable, it has been around for decades in various incarnations. I have never noticed bit rot in this system, though I imagine it could happen if a disk is gradually going bad. Please correct me if I'm wrong because I don't want to spread incorrect information. A couple of months ago I read a couple of reports of filesystem corruption on OpenBSD. I didn't have time to investigate deeply and I don't know if these issues were even real. Even if they were real I don't know if the problem was due to user error or a defect in the OS. Does anybody know anything about this? That's why multiple backups help. Agreed. See below. You might want to set up a raid5 backup, as this detects parity errors. More complicated though. This is exactly the kind of reason that hybrid volume management systems + filesystems such as Btrfs and ZFS have become popular. I do not know anything about OpenBSD's LVM. One weakness in such as system (ask me how I know!) is that if the NAS goes gradually bad, the errors will propagate to the backup. Using rsync without the --delete option most of the time alleviates this somewhat. Only run with --delete when the backup starts getting full and you are confident that your NAS drive is ok. This is an excellent reason for implementing a system that includes not only backups, but long term storage /archives/ too. Andrew -- OpenPGP key: EB28 0338 28B7 19DA DAB0 B193 D21D 996E 883B E5B9
Re: Home NAS
I have done similar things on Linux for years and am now doing them on OpenBSD. Sounds like what you want to do can be done with a simple rsync script. OpenBSD ffs (ufs) should be stable, it has been around for decades in various incarnations. I have never noticed bit rot in this system, though I imagine it could happen if a disk is gradually going bad. That's why multiple backups help. You might want to set up a raid5 backup, as this detects parity errors. More complicated though. One weakness in such as system (ask me how I know!) is that if the NAS goes gradually bad, the errors will propagate to the backup. Using rsync without the --delete option most of the time alleviates this somewhat. Only run with --delete when the backup starts getting full and you are confident that your NAS drive is ok. Dave On 11/14/19, Jan Betlach wrote: > > Hi guys, > > I am setting up a home NAS for five users. Total amount of data stored > on NAS will not exceed 5 TB. > Clients are Macs and OpenBSD machines, so that SSHFS works fine from > both (no need for NFS or Samba). > I am much more familiar and comfortable with OpenBSD than with FreeBSD. > My dilema while stating the above is as follows: > > Will the OpenBSD’s UFS stable and reliable enough for intended > purpose? NAS will consist of just one encrypted drive, regularly backed > to hardware RAID encrypted two-disks drive via rsync. > > Should I byte the bullet and build the NAS on FreeBSD taking advantage > of ZFS, snapshots, replications, etc? Or is this an overkill? > > BTW my most important data is also backed off-site. > > Thank you in advance for your comments. > > Jan > > -- David J. Raymond david.raym...@nmt.edu http://physics.nmt.edu/~raymond
Re: Home NAS
On Thu, 14 Nov 2019, Jan Betlach wrote: Should I byte the bullet and build the NAS on FreeBSD taking advantage of ZFS, snapshots, replications, etc? Or is this an overkill? I built my "NAS" with FreeBSD due to the self healing properties of ZFS with checksums and redundant data, and due to the fact that ZFS became more or less a standard that run in many OS and will not be forgotten from one day to other. But I am unhappy with ZFS, it is too heavy and complicated for my purpose, do not pass to the unix environement, and my use is for less than NAS, "only" for archival purposes. But I do not see at the moment a simple alternative that fulfill my expectations. Rodrigo
Re: Home NAS
On Thu, Nov 14, 2019 at 3:29 PM Jan Betlach wrote: > > Hi guys, > > I am setting up a home NAS for five users. Total amount of data stored > on NAS will not exceed 5 TB. > Clients are Macs and OpenBSD machines, so that SSHFS works fine from > both (no need for NFS or Samba). > I am much more familiar and comfortable with OpenBSD than with FreeBSD. > My dilema while stating the above is as follows: > > Will the OpenBSD’s UFS stable and reliable enough for intended > purpose? NAS will consist of just one encrypted drive, regularly backed > to hardware RAID encrypted two-disks drive via rsync. > > Should I byte the bullet and build the NAS on FreeBSD taking advantage > of ZFS, snapshots, replications, etc? Or is this an overkill? > > BTW my most important data is also backed off-site. > > Thank you in advance for your comments. > > See https://www.vincentdelft.be/post/post_20190203 (slides here https://www.vincentdelft.be/static/post/post_20190203/openbsd-as-nas.pdf and video recording here https://archive.fosdem.org/2019/schedule/event/openbsd_nas_system/ ) > Jan > > -- eduardominguez.es
