Re: mdadm usage

2021-01-02 Thread Linux-Fan
deloptes writes: Linux-Fan wrote: > deloptes writes: >> Andrei POPESCU wrote: >> >> Each LSI card has a 6 bay cage attached and I have raided 6x2TB WD RED >> >> spinning discs (for data) and 2x1TB WD RED spinning discs (for OS) [...] > Sounds OK to me :) From my point of view, I would work

Re: mdadm usage

2021-01-02 Thread deloptes
Linux-Fan wrote: > deloptes writes: > >> Andrei POPESCU wrote: >> >> >> Each LSI card has a 6 bay cage attached and I have raided 6x2TB WD RED >> >> spinning discs (for data) and 2x1TB WD RED spinning discs (for OS) >> > >> > 1TB for OS (assuming RAID1) seems... excessive to me. All my current

Re: mdadm usage

2021-01-02 Thread Linux-Fan
deloptes writes: Andrei POPESCU wrote: >> Each LSI card has a 6 bay cage attached and I have raided 6x2TB WD RED >> spinning discs (for data) and 2x1TB WD RED spinning discs (for OS) > > 1TB for OS (assuming RAID1) seems... excessive to me. All my current > installations are in 10 GiB

Re: mdadm usage

2021-01-02 Thread Jesper Dybdal
On 2020-12-30 14:58, Andy Smith wrote: You can see the event count with: # mdadm --examine /dev/sda1 # or whatever the member device is So yes in one way your idea that the most recently modified half is the one chosen could be said to be correct, if by "most recently modified" you actually

Re: mdadm usage

2021-01-02 Thread The Wanderer
On 2021-01-02 at 07:09, Andrei POPESCU wrote: > On Sb, 02 ian 21, 13:49:27, Andrei POPESCU wrote: > >> On Sb, 02 ian 21, 06:28:24, The Wanderer wrote: >>> On previous systems I investigated things like autoclean, but >>> AFAIR I never identified a way to get such a mechanism to keep >>> the

Re: mdadm usage

2021-01-02 Thread The Wanderer
On 2021-01-02 at 06:49, Andrei POPESCU wrote: > On Sb, 02 ian 21, 06:28:24, The Wanderer wrote: > >> On 2021-01-02 at 05:58, Andrei POPESCU wrote: >> >>> How many package versions do you keep around? >>> My laptop is indeed on a 30 GiB partition, but using only 14 GiB. >>> I do (auto)clean the

Re: mdadm usage

2021-01-02 Thread Andrei POPESCU
On Sb, 02 ian 21, 13:49:27, Andrei POPESCU wrote: > On Sb, 02 ian 21, 06:28:24, The Wanderer wrote: > > > > At the very least, I'd need to be able to keep the .deb files for > > whatever version is presently installed - and probably more than that, > > given that I do sometimes try installing a

Re: mdadm usage

2021-01-02 Thread Andrei POPESCU
On Sb, 02 ian 21, 06:28:24, The Wanderer wrote: > On 2021-01-02 at 05:58, Andrei POPESCU wrote: > > > How many package versions do you keep around? > > Basically all of them. In theory I go in and delete the oldest unneeded > ones from time to time, but in practice that hasn't happened much. >

Re: mdadm usage

2021-01-02 Thread deloptes
Andrei POPESCU wrote: > My laptop is indeed on a 30 GiB partition, but using only 14 GiB. I do > (auto)clean the package cache more or less regularly though, even on > unstable installs. Hi Andrei, thank you for sharing your thoughts. # df -hl . Filesystem Size Used Avail Use% Mounted on

Re: mdadm usage

2021-01-02 Thread The Wanderer
On 2021-01-02 at 05:58, Andrei POPESCU wrote: > On Sb, 02 ian 21, 04:35:59, The Wanderer wrote: > >> / itself (excluding child filesystems) contains 23GB of data. 22GB >> of that is under /root. 17GB of *that* consists of backups of >> other data that isn't read-time-accessible to any other user

Re: mdadm usage

2021-01-02 Thread Andrei POPESCU
On Sb, 02 ian 21, 04:35:59, The Wanderer wrote: > > / itself (excluding child filesystems) contains 23GB of data. 22GB of > that is under /root. 17GB of *that* consists of backups of other > data that isn't read-time-accessible to any other user - both because it > isn't convenient to create the

Re: mdadm usage

2021-01-02 Thread Andrei POPESCU
On Sb, 02 ian 21, 10:05:11, deloptes wrote: > Andrei POPESCU wrote: > > > > The speed gain of SSD vs. spinning discs for the OS is hard to describe. > > Think jet aircraft vs. car. > > > > I've done this for a laptop (partially out of necessity, after I dropped > > it) and it was like buying a

Re: mdadm usage

2021-01-02 Thread The Wanderer
On 2021-01-02 at 03:36, Andrei POPESCU wrote: > On Vi, 01 ian 21, 23:09:01, deloptes wrote: >> >> Each LSI card has a 6 bay cage attached and I have raided 6x2TB WD >> RED spinning discs (for data) and 2x1TB WD RED spinning discs (for >> OS) > > 1TB for OS (assuming RAID1) seems... excessive to

Re: mdadm usage

2021-01-02 Thread deloptes
Andrei POPESCU wrote: >> Each LSI card has a 6 bay cage attached and I have raided 6x2TB WD RED >> spinning discs (for data) and 2x1TB WD RED spinning discs (for OS) > > 1TB for OS (assuming RAID1) seems... excessive to me. All my current > installations are in 10 GiB partitions with only a

Re: mdadm usage

2021-01-02 Thread Andrei POPESCU
On Vi, 01 ian 21, 23:09:01, deloptes wrote: > > Each LSI card has a 6 bay cage attached and I have raided 6x2TB WD RED > spinning discs (for data) and 2x1TB WD RED spinning discs (for OS) 1TB for OS (assuming RAID1) seems... excessive to me. All my current installations are in 10 GiB partitions

Re: mdadm usage

2021-01-01 Thread Richard Hector
On 31/12/20 7:29 am, Marc Auslander wrote: IMHO, there are two levels of backup. The more common use is to undo user error - deleting the wrong thing or changing something and wanting to back out. For that, backups on the same system are the most convenient. And if its on the same system, and

Re: mdadm usage

2021-01-01 Thread deloptes
Andy Smith wrote: > That is a really strange comment to me. No SSDs have batteries. > Almost no RAID cards have batteries anymore. Supercapacitors have > obsoleted the battery for such purposes. All SSD power loss > protection is supercaps. And if you try to buy a modern RAID card > with a BBU

Re: mdadm usage

2020-12-31 Thread Andy Smith
Hello, On Thu, Dec 31, 2020 at 06:18:29PM +0300, Reco wrote: > On Thu, Dec 31, 2020 at 03:06:34PM +, Andy Smith wrote: > > Datasheet says: > > > > * Enhanced Power-Loss Data Protection with Tantal capacitors > > It does not have a battery = it does not have a BBU. > Samsung can dance around

Re: mdadm usage

2020-12-31 Thread Reco
On Thu, Dec 31, 2020 at 03:06:34PM +, Andy Smith wrote: > On Thu, Dec 31, 2020 at 05:42:35PM +0300, Reco wrote: > > It's a cheap model (relatively), and lack all those fancy BBU features, > > but it works for my employer: > > > > # smartctl -a /dev/sdj > > ... > > Vendor:

Re: mdadm usage

2020-12-31 Thread Andy Smith
Hello, On Thu, Dec 31, 2020 at 05:42:35PM +0300, Reco wrote: > It's a cheap model (relatively), and lack all those fancy BBU features, > but it works for my employer: > > # smartctl -a /dev/sdj > ... > Vendor: SAMSUNG > Product: MZILT1T9HAJQ/007 Datasheet says: *

Re: mdadm usage

2020-12-31 Thread Reco
Hi. On Wed, Dec 30, 2020 at 11:14:37PM +0100, deloptes wrote: > Can someone recommend server or NAS grade (SATA) SSD - a reliable one > for RAID use? It's a cheap model (relatively), and lack all those fancy BBU features, but it works for my employer: # smartctl -a /dev/sdj ... Vendor:

Re: mdadm usage

2020-12-31 Thread Andy Smith
Hello, On Thu, Dec 31, 2020 at 09:17:03AM -0500, Michael Stone wrote: > On Thu, Dec 31, 2020 at 07:25:54AM -0500, rhkra...@gmail.com wrote: > >What do you mean by power loss protection -- do you mean, for example, that > >the host computer is on a UPS, or is that a feature of some SSDs? > > It's

Re: mdadm usage

2020-12-31 Thread Michael Stone
On Thu, Dec 31, 2020 at 07:25:54AM -0500, rhkra...@gmail.com wrote: What do you mean by power loss protection -- do you mean, for example, that the host computer is on a UPS, or is that a feature of some SSDs? It's a feature of server SSDs. I wouldn't worry about it on a consumer device,

Re: mdadm usage

2020-12-31 Thread rhkramer
On Wednesday, December 30, 2020 06:20:09 PM Andy Smith wrote: > On Wed, Dec 30, 2020 at 11:14:37PM +0100, deloptes wrote: > > Can someone recommend server or NAS grade (SATA) SSD - a reliable one for > > RAID use? ... > Then make sure it has power loss protection. What do you mean by power loss

Re: mdadm usage

2020-12-30 Thread Andy Smith
Hello, On Wed, Dec 30, 2020 at 11:14:37PM +0100, deloptes wrote: > Can someone recommend server or NAS grade (SATA) SSD - a reliable one for > RAID use? I suggest checking smartctl values on your existing device to see how much data you write in a week or so, then convert that to the matching

Re: mdadm usage

2020-12-30 Thread Nicholas Geovanis
> On Wed, Dec 30, 2020, 3:08 PM deloptes wrote: > >> Thomas A. Anderson wrote: >> >> > If hardware raid (like if I bought a controller), would it be any >> > different, if I removed the drives and just put on one another machine >> > -- would I be able to see the data on it like a normal drive?

Re: mdadm usage

2020-12-30 Thread deloptes
Michael Stone wrote: > The improvement in seek times typically makes for a dramatic improvement > in usability and user experience, regardless of maximum transfer rate. > Replacing an SSD with an HD will usually breathe new life into an old > system; people tend to dramatically underestimate how

Re: mdadm usage

2020-12-30 Thread Michael Stone
On Wed, Dec 30, 2020 at 09:44:09PM +0100, deloptes wrote: It depends. For example on the machine at home with LSI adapter that provides the speed of SATA II I do not see any benefit of using SSD except power saving The improvement in seek times typically makes for a dramatic improvement in

Re: mdadm usage

2020-12-30 Thread deloptes
Thomas A. Anderson wrote: > If hardware raid (like if I bought a controller), would it be any > different, if I removed the drives and just put on one another machine > -- would I be able to see the data on it like a normal drive? Or would I > run into the same issue?? if you choose using

Re: mdadm usage

2020-12-30 Thread deloptes
Nicholas Geovanis wrote: > And also an argument in favor of using only SSD, now that we're past the > first couple generations of it which were quite unreliable. It depends. For example on the machine at home with LSI adapter that provides the speed of SATA II I do not see any benefit of using

Re: mdadm usage

2020-12-30 Thread Nicholas Geovanis
On Wed, Dec 30, 2020, 1:12 PM Andrei POPESCU wrote: > On Mi, 30 dec 20, 13:29:05, Marc Auslander wrote: > > > > IMHO, there are two levels of backup. The more common use is to undo > > user error - deleting the wrong thing or changing something and wanting > > to back out. For that, backups on

Re: mdadm usage

2020-12-30 Thread Michael Stone
On Wed, Dec 30, 2020 at 09:12:08PM +0200, Andrei POPESCU wrote: On Mi, 30 dec 20, 13:29:05, Marc Auslander wrote: IMHO, there are two levels of backup. The more common use is to undo user error - deleting the wrong thing or changing something and wanting to back out. For that, backups on the

Re: mdadm usage

2020-12-30 Thread Andrei POPESCU
On Mi, 30 dec 20, 13:29:05, Marc Auslander wrote: > > IMHO, there are two levels of backup. The more common use is to undo > user error - deleting the wrong thing or changing something and wanting > to back out. For that, backups on the same system are the most > convenient. And if its on the

Re: mdadm usage

2020-12-30 Thread Marc Auslander
Reco writes: > >And what purpose would it serve? IMO it's not a backup unless it's >stored in a way that's inaccessible to the system its taken from (until >it's actually needed of course). > >Reco IMHO, there are two levels of backup. The more common use is to undo user error - deleting the

Re: mdadm usage

2020-12-30 Thread Andrei POPESCU
On Mi, 30 dec 20, 16:56:25, mick crane wrote: > > I just confused myself. Initially I read somewhere that to make the raid > first copy the OS from one disk to another. Do you mean copy as in 'cp' or 'rsync' or similar? RAID operates at a lower level, typically below the filesystem (except for

Re: mdadm usage

2020-12-30 Thread Reco
Hi. On Wed, Dec 30, 2020 at 09:55:06AM -0500, Marc Auslander wrote: > Andrei POPESCU writes: > > > >Automatic mirroring / synchronizing is unsuitable for backups, because > >it will also sync accidental changes to files (including deletions) or > >filesystem corruptions in case of

Re: mdadm usage

2020-12-30 Thread Dan Ritter
Marc Auslander wrote: > Andrei POPESCU writes: > > > >Automatic mirroring / synchronizing is unsuitable for backups, because > >it will also sync accidental changes to files (including deletions) or > >filesystem corruptions in case of power outage or system crash (that may > >lead to

Re: mdadm usage

2020-12-30 Thread mick crane
On 2020-12-30 13:38, Andy Smith wrote: Hi Mick, On Tue, Dec 29, 2020 at 03:32:07PM +, mick crane wrote: On 2020-12-29 13:10, Andy Smith wrote: >The default metadata format (v1.2) for mdadm is at the beginning of >the device. If you've put a filesystem directly on the md device >then the

Re: mdadm usage

2020-12-30 Thread Marc Auslander
Andrei POPESCU writes: > >Automatic mirroring / synchronizing is unsuitable for backups, because >it will also sync accidental changes to files (including deletions) or >filesystem corruptions in case of power outage or system crash (that may >lead to corrupted files or entire directories

Re: mdadm usage

2020-12-30 Thread Nicholas Geovanis
On Wed, Dec 30, 2020, 8:03 AM Nicholas Geovanis wrote: > > > On Wed, Dec 30, 2020, 7:39 AM Jesper Dybdal > wrote: > >> I would hope that the most recently modified half of the array would be >> the one to overwrite the least recently modified one, so that a >> temporary absence of one disk

mdadm usage

2020-12-30 Thread Nicholas Geovanis
On Wed, Dec 30, 2020, 7:39 AM Jesper Dybdal wrote: > I would hope that the most recently modified half of the array would be > the one to overwrite the least recently modified one, so that a > temporary absence of one disk which later comes back unmodified, will > not destroy data. > > Is that

Re: mdadm usage

2020-12-30 Thread Andy Smith
Hello, On Wed, Dec 30, 2020 at 02:38:46PM +0100, Jesper Dybdal wrote: > I would hope that the most recently modified half of the array would be the > one to overwrite the least recently modified one, so that a temporary > absence of one disk which later comes back unmodified, will not destroy >

Re: mdadm usage

2020-12-30 Thread Andy Smith
On Wed, Dec 30, 2020 at 11:42:37AM +0100, Thomas A. Anderson wrote: > When i enter mdadm --examine /dev/sdb > > I get: > > /dev/sdb: > >     MBR Magic: aa55 > > Partition[0] : 3907026944 sectors at         2048 (type 83) It would say more than that if sdb had ever been an md RAID member. Are

Re: mdadm usage

2020-12-30 Thread The Wanderer
On 2020-12-30 at 08:38, Andy Smith wrote: > Hi Mick, > > On Tue, Dec 29, 2020 at 03:32:07PM +, mick crane wrote: > >> On 2020-12-29 13:10, Andy Smith wrote: >> >>> The default metadata format (v1.2) for mdadm is at the beginning >>> of the device. If you've put a filesystem directly on the

Re: mdadm usage

2020-12-30 Thread Jesper Dybdal
On 2020-12-30 02:21, Michael Stone wrote: On Wed, Dec 30, 2020 at 01:53:59AM +0100, deloptes wrote: you can start one of the drives that was member of raid1 array on any computer, just as Reco said by assembling the mdraid on that computer. Just be very careful about ever putting both

Re: mdadm usage

2020-12-30 Thread Andy Smith
Hi Mick, On Tue, Dec 29, 2020 at 03:32:07PM +, mick crane wrote: > On 2020-12-29 13:10, Andy Smith wrote: > >The default metadata format (v1.2) for mdadm is at the beginning of > >the device. If you've put a filesystem directly on the md device > >then the presence of the metadata will

Re: mdadm usage

2020-12-30 Thread Alexander V. Makartsev
On 30.12.2020 15:42, Thomas A. Anderson wrote: it could all very well be, that I have borked these two drives. It's not the end of the world, there was no data loss, and the data was already transferred off of them. And, my lesson has been learned. RAID sounds all good and dandy, and does

Re: mdadm usage

2020-12-30 Thread Dan Ritter
Thomas A. Anderson wrote: > Which bring me to my final question, just for closure. > > If hardware raid (like if I bought a controller), would it be any > different, if I removed the drives and just put on one another machine > -- would I be able to see the data on it like a normal drive? Or

Re: mdadm usage

2020-12-30 Thread Thomas A. Anderson
When i enter mdadm --examine /dev/sdb I get: /dev/sdb:     MBR Magic: aa55 Partition[0] : 3907026944 sectors at         2048 (type 83) So I thought I was good. I then tried to reassemble: mdadm -A -R /dev/md0 /dev/sdb   I get: mdadm: cannot assembler mbr metadata on /dev/sdb So, I tried

Re: mdadm usage

2020-12-30 Thread deloptes
Michael Stone wrote: > Just be very careful about ever putting both halves of the array on the > same computer again, as one will overwrite the other, potentially > automatically, and not necessarily in a direction you like. yes this is true. when you add the device you should mark it as faulty

Re: mdadm usage

2020-12-30 Thread Andrei POPESCU
On Ma, 29 dec 20, 21:37:01, Thomas A. Anderson wrote: > > I have been using it for years, and while not a bad "thing," in > retrospect, not sure it actually meant my critieria (eh, who knows, > maybe it did), but now I have a more clear use case, basically to have a > clean backup. I was using

Re: mdadm usage

2020-12-29 Thread Dan Ritter
basti wrote: > On a file server in a production environment I would prefer raid 6. > Making statements so cavalierly is a bad idea. There are excellent reasons for preferring RAID 1, 10, 6, 60, and various ZFSisms, according to expected requirements. -dsr-

Re: mdadm usage

2020-12-29 Thread Michael Stone
On Wed, Dec 30, 2020 at 01:53:59AM +0100, deloptes wrote: you can start one of the drives that was member of raid1 array on any computer, just as Reco said by assembling the mdraid on that computer. Just be very careful about ever putting both halves of the array on the same computer again,

Re: mdadm usage

2020-12-29 Thread deloptes
Thomas A. Anderson wrote: > Once I can get anything off one of these two drives, I will then switch > my current setup to 1 drive (8TB), with an attached 8TB drive that will > be be backup --weekly, or whatever. you can start one of the drives that was member of raid1 array on any computer, just

Re: mdadm usage

2020-12-29 Thread Linux-Fan
Thomas A. Anderson writes: Thanks Reco, yeah that RAID6 looks pretty robust. I will read more about it in the future. When I type cat /proc/mdstat, I get: Personalities: unused devices: Is that all there is? Because to me this means: RAID kernel driver loaded but no arrays running

Re: mdadm usage

2020-12-29 Thread Thomas A. Anderson
Thanks Reco, yeah that RAID6 looks pretty robust. I will read more about it in the future. When I type cat /proc/mdstat, I get: Personalities: unused devices: Thanks Deloptes,  the Borg backup looks very promising and I think it might actually best fit my needs. I will check it out! Thanks

Re: mdadm usage

2020-12-29 Thread Alexander V. Makartsev
On 29.12.2020 20:32, mick crane wrote: On 2020-12-29 13:10, Andy Smith wrote: <..> The default metadata format (v1.2) for mdadm is at the beginning of the device. If you've put a filesystem directly on the md device then the presence of the metadata will prevent it being recognised as a simple

Re: mdadm usage

2020-12-29 Thread mick crane
On 2020-12-29 13:10, Andy Smith wrote: <..> The default metadata format (v1.2) for mdadm is at the beginning of the device. If you've put a filesystem directly on the md device then the presence of the metadata will prevent it being recognised as a simple filesystem. What you can do is force

Re: mdadm usage

2020-12-29 Thread Andy Smith
Hello, On Tue, Dec 29, 2020 at 07:58:50AM +0100, Thomas A. Anderson wrote: > I have been "using" mdadm to run software raid1 (stripping) on a file > server i have been running. As others have noted, RAID-1 is not striping but mirroring. I'll assume you have used RAID-1. Showing us the content of

Re: mdadm usage

2020-12-29 Thread basti
On a file server in a production environment I would prefer raid 6. Am 29.12.20 um 08:45 schrieb Felix Miata: > Thomas A. Anderson composed on 2020-12-29 07:58 (UTC+0100): > >> I have been "using" mdadm to run software raid1 (stripping) on a file >> server i have been running. > > RAID 1 is two

Re: mdadm usage

2020-12-29 Thread deloptes
Reco wrote: > >> Maybe I just need two drives, and just do a cron job to rysnc to backup >> once a week or something. > > RAID is not a substitute for a backup. Backup is not a substitute for > mirroring. One does not exclude another, so use both. I find borg backup worth looking in,

Re: mdadm usage

2020-12-28 Thread Felix Miata
Thomas A. Anderson composed on 2020-12-29 07:58 (UTC+0100): > I have been "using" mdadm to run software raid1 (stripping) on a file > server i have been running. RAID 1 is two devices that are mirrors of each other, redundancy, with some loss in speed. Loose one device, and you still have all

Re: mdadm usage

2020-12-28 Thread Reco
Hi. On Tue, Dec 29, 2020 at 07:58:50AM +0100, Thomas A. Anderson wrote: > I have been "using" mdadm to run software raid1 (stripping) on a file > server i have been running. RAID1 is mirroring. Stripping is RAID0. RAID1 provides redundancy. RAID0 does not. > It is only now that I

mdadm usage

2020-12-28 Thread Thomas A. Anderson
Hello, I have been "using" mdadm to run software raid1 (stripping) on a file server i have been running. None of the drives have failed, and I have even setup subsequent drives in the same scenario. It is only now that I wonder if I am even using RAID1 properly? In other words, now that I try