Re: [gentoo-user] SDD strategies...
(Hmm, weird, this failed to send and was laying around in my outbox.)
Am Samstag, 21. März 2020, 16:29:48 CET schrieb David Haller:
> Hello,
>
> On Sat, 21 Mar 2020, Marc Joliet wrote:
> >Am Mittwoch, 18. März 2020, 16:56:52 CET schrieb antlists:
> [..]
>
> >> Can't remember where it was - some mag ran a stress-test on a bunch of
> >> SSDs and they massively outlived their rated lives ... I think even the
> >> first to fail survived about 18months of continuous hammering - and I
> >> mean hammering!
> >
> >The German c't magazine did a similar test of various SSDs from
> >different
>
> I mentioned that in the other thread ("SDD, what features..."), I plan
> to sum up the articles tomorrow. I'd guess he means that test ;)
>
> >price categories, and they all showed the same result (I think some
> >exceeded their lifetime by more than a factor of two, and the minimum was
> >something like 1.5).
>
> If you mean TBW by "lifetime": All above factor 2, best: over 18. Ok,
> those were the "brand models" (1 Crucial, 1 OCZ (Toshiba), 2 Samsung
> and 2 Sandisk, and 2 drives of each model)...
Yes, that's what I meant (the SSDs are rated for an expected amount of data
written they can sustain, and apparently it's estimated very conservatively).
> Fun fact: one of the test PCs died and killed two of the 3 remaining
> SSDs, the second Sandisk Extreme Pro (the first had died already) and
> the first Samsung 850 Pro. The remaining second Samsung 850 Pro in the
> other Test-PC was still being hammered 4.5 months later with 8 PeBi
> written (all drives were 240/256 GB), but showing first "Uncorrectable
> Errors" via SMART.
>
> The test also included the failure mode as well, e.g. "dead as a
> brick" in a moment, warning signs via SMART, failure to write but
> still readable etc.
>
> The mag followed that up with a test of el-cheapo SSDs ... which I'll
> include in my summary.
Cool, I forgot most of the details (including that the test was split between
brand and cheapo models), so I'm looking forward to your summary!
> -dnh
Greetings
--
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup
signature.asc
Description: This is a digitally signed message part.
Re: [gentoo-user] SDD strategies...
Frank Steinmetzger wrote: > On Tue, Mar 17, 2020 at 05:13:52AM -0500, Dale wrote: >> Neil Bothwick wrote: >>> On Tue, 17 Mar 2020 10:43:58 +0100, Andrea Conti wrote: >>> >> The SSD is currently reporting 98% of its rated life left: I feel >> quite confident it's going to outlast the laptop's useful life. > What are you using to get that niformation? smartctl -A /dev/sdX >>> 83% after five years of recompiling LO and Chromium, not bad :) >>> >>> >> Dang. That makes me feel better about getting one now. Heck, I'm using >> a 160GB drive currently for my OS. > A hard disk that small is probably also very old, and thus slow. > > When I was still a poor little student, I used a 500 Gig WD disk as main > drive for everything. It came originally from one of my first external > enclosures. At some point it started to exhibit failures. > > By then I was earning my own dough. So I got me a brand-new WD Blue 1 TB. > That drive had a speed increase of 150(!) %. IOW, it was 2½ times as fast as > the old one. I still have the screenshots from HD Tune from back then. > Imagine the speed-up you will get even from an up-to-date harddisk compared > to your old veteran. > True. I've seen that in the past too. The drive is a WD black which I think at the time was a server grade type thing. It's actually pretty dang fast. Someone on this list wanted to get rid of it so they shipped it to me. Hey, it works. It's fairly fast. Odds are, if I get a SSD and later build a NAS rig, it'll go in that. I tend to use drives until they break, in something. ;-) Still, even if the drive was new, a SSD would still be faster. That's the theory at least. lol Dale :-) :-)
Re: [gentoo-user] SDD strategies...
On Tue, Mar 17, 2020 at 05:13:52AM -0500, Dale wrote: > Neil Bothwick wrote: > > On Tue, 17 Mar 2020 10:43:58 +0100, Andrea Conti wrote: > > > The SSD is currently reporting 98% of its rated life left: I feel > quite confident it's going to outlast the laptop's useful life. > >>> What are you using to get that niformation? > >> smartctl -A /dev/sdX > > 83% after five years of recompiling LO and Chromium, not bad :) > > > > > > Dang. That makes me feel better about getting one now. Heck, I'm using > a 160GB drive currently for my OS. A hard disk that small is probably also very old, and thus slow. When I was still a poor little student, I used a 500 Gig WD disk as main drive for everything. It came originally from one of my first external enclosures. At some point it started to exhibit failures. By then I was earning my own dough. So I got me a brand-new WD Blue 1 TB. That drive had a speed increase of 150(!) %. IOW, it was 2½ times as fast as the old one. I still have the screenshots from HD Tune from back then. Imagine the speed-up you will get even from an up-to-date harddisk compared to your old veteran. -- Gruß | Greetings | Qapla’ Please do not share anything from, with or about me on any social network. I have a closet full of nothing to wear. signature.asc Description: PGP signature
Re: [gentoo-user] SDD strategies...
Hello,
On Sat, 21 Mar 2020, Marc Joliet wrote:
>Am Mittwoch, 18. März 2020, 16:56:52 CET schrieb antlists:
[..]
>> Can't remember where it was - some mag ran a stress-test on a bunch of
>> SSDs and they massively outlived their rated lives ... I think even the
>> first to fail survived about 18months of continuous hammering - and I
>> mean hammering!
>
>The German c't magazine did a similar test of various SSDs from
>different
I mentioned that in the other thread ("SDD, what features..."), I plan
to sum up the articles tomorrow. I'd guess he means that test ;)
>price categories, and they all showed the same result (I think some exceeded
>their lifetime by more than a factor of two, and the minimum was something
>like 1.5).
If you mean TBW by "lifetime": All above factor 2, best: over 18. Ok,
those were the "brand models" (1 Crucial, 1 OCZ (Toshiba), 2 Samsung
and 2 Sandisk, and 2 drives of each model)...
Fun fact: one of the test PCs died and killed two of the 3 remaining
SSDs, the second Sandisk Extreme Pro (the first had died already) and
the first Samsung 850 Pro. The remaining second Samsung 850 Pro in the
other Test-PC was still being hammered 4.5 months later with 8 PeBi
written (all drives were 240/256 GB), but showing first "Uncorrectable
Errors" via SMART.
The test also included the failure mode as well, e.g. "dead as a
brick" in a moment, warning signs via SMART, failure to write but
still readable etc.
The mag followed that up with a test of el-cheapo SSDs ... which I'll
include in my summary.
-dnh
--
| Ceci n'est pas une pipe
Re: [gentoo-user] SDD strategies...
Am Mittwoch, 18. März 2020, 16:56:52 CET schrieb antlists: > On 17/03/2020 05:59, tu...@posteo.de wrote: > > Hi, > > > > currentlu I am setting up a new PC for my 12-years old one, > > which has reached the limits of its "computational power" :) > > > > SSDs are a common replacement for HDs nowaday -- but I still trust my > > HDs more than this "flashy" things...call me retro or oldschool, but > > it my current "Bauchgefühl" (gut feeling). It really is an art to know when to trust your gut feeling :-) . Thankfully in this case we have data! > Can't remember where it was - some mag ran a stress-test on a bunch of > SSDs and they massively outlived their rated lives ... I think even the > first to fail survived about 18months of continuous hammering - and I > mean hammering! The German c't magazine did a similar test of various SSDs from different price categories, and they all showed the same result (I think some exceeded their lifetime by more than a factor of two, and the minimum was something like 1.5). > > To reduce write cycles to the SSD, which are quite a lot when using > > UNIX/Limux (logging etc) and especially GENTOO (compiling sources > > instead of using binary packages -- which is GOOD!), I am planning > > the following setup: > > > > The sustem will boot from SSD. > > > > The HD will contain the whole system including the complete root > > filesustem. Updateing, installing via Gentoo tools will run using > > the HD. If that process has ended, I will rsync the HD based root > > fileystem to the SSD. > > Whatever for? Yeah, I did the $PORTAGE_TMPDIR on tmpfs thing for a while, but I don't feel like "wasting" the RAM of my Gentoo systems in that way anymore. And guess what: # smartctl -x /dev/sda | grep -i lifetime_remain 202 Percent_Lifetime_Remain CK 092 092 001-8 This is for an SSD (Crucial MX500) that I've been using for about 1.5 years (since early November 2018), and which hosts the entirety of / (including /var and /home), only my media FS resides on HDDs. The Crucial SSD I used before that (128 GB) was at 95 % last I checked, and I had been using that for about four years (it's laying on a shelf now for installation in my home server when I find the time for that). (Oh, and everything is on BTRFS.) > > Folders, which will be written to by the sustem while running will > > be symlinked to the HD. > > > > This should work...? > > > > Or is there another idea to setup a system which will benefit from > > the advantages of a SSD by avoiding its disadvantages? > > If you've got both an SSD and an HD, just use the HD for swap, /tmp, > /var/tmp/portage (possibly the whole of /var/tmp), and any other area > where you consider files to be temporary. As I mentioned above, I'm not even doing that. I did until about 2017, but stopped because it increased the load on my desktop too much for it to be usable during upgrades (poor old thing). > > Background: I am normally using a PC a long time and try to avoid > > buying things for reasons like being more modern or being newer. Keep in mind we're talking about technology that is pretty old by now. IIUC, what's new now is mainly that it's not niche anymore (see https:// en.wikipedia.org/wiki/Solid-state_drive#Development_and_history). > > Any idea to setup such a sustem is heardly welcone -- thank you > > very much in advance! > > Why waste time and effort for a complex setup when it's going to gain > you bugger all. I agree, I don't think a complicated setup like that is worth the effort. I was really glad about being able to put the whole / on my SSD after I got a bigger one, just because of how it simplified things. > The only thing I would really advise for is that (a) you think about > some form of journalling - LVM or btrfs - for your root file-system to > protect against a messed up upgrade - take a snapshot, upgrade, and if > anything goes wrong it's an easy roll-back. > > Likewise, do the same for the rotating rust, and use that to back up > /home - you can use some option to rsync that only over-writes what's > changed, so you do a "snapshot then back up" and have loads of backups > going back however far ... Backups are something you should be doing anyway. Even a local backup is better than no backup at all. You won't miss your data until it's gone, and then you'll *really* miss it. > Cheers, > Wol Greetings -- Marc Joliet -- "People who think they know everything really annoy those of us who know we don't" - Bjarne Stroustrup signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] SDD strategies...
antlists wrote: > On 17/03/2020 11:54, madscientistatlarge wrote: >> The issue is not usually end of trusted life, but rather random >> failure. I've barely managed to recover failed hard drives, That is >> less likely on SSD though possibly less likely to happen. > > The drive may be less likely to fail, but I'd say raid or backups are > a necessity. > > From what I've heard, SSDs tend to go read-only when they fail (that's > fine), BUT SELF-DESTRUCT ON A POWER CYCLE!!! > > So don't bank on being able to access a failed SSD after a reboot. > > Cheers, > Wol > > That's something interesting to know. If one fails, recover but don't reboot. Knowing that is awesome. It may not be a guarantee but why risk it? At the least, do a backup first, then reboot if needed. This old guy needs to remember that. Dale :-) :-)
Re: [gentoo-user] SDD strategies...
On 17/03/2020 05:59, tu...@posteo.de wrote: Hi, currentlu I am setting up a new PC for my 12-years old one, which has reached the limits of its "computational power" :) SSDs are a common replacement for HDs nowaday -- but I still trust my HDs more than this "flashy" things...call me retro or oldschool, but it my current "Bauchgefühl" (gut feeling). Can't remember where it was - some mag ran a stress-test on a bunch of SSDs and they massively outlived their rated lives ... I think even the first to fail survived about 18months of continuous hammering - and I mean hammering! To reduce write cycles to the SSD, which are quite a lot when using UNIX/Limux (logging etc) and especially GENTOO (compiling sources instead of using binary packages -- which is GOOD!), I am planning the following setup: The sustem will boot from SSD. The HD will contain the whole system including the complete root filesustem. Updateing, installing via Gentoo tools will run using the HD. If that process has ended, I will rsync the HD based root fileystem to the SSD. Whatever for? Folders, which will be written to by the sustem while running will be symlinked to the HD. This should work...? Or is there another idea to setup a system which will benefit from the advantages of a SSD by avoiding its disadvantages? If you've got both an SSD and an HD, just use the HD for swap, /tmp, /var/tmp/portage (possibly the whole of /var/tmp), and any other area where you consider files to be temporary. Background: I am normally using a PC a long time and try to avoid buying things for reasons like being more modern or being newer. Any idea to setup such a sustem is heardly welcone -- thank you very much in advance! Why waste time and effort for a complex setup when it's going to gain you bugger all. The only thing I would really advise for is that (a) you think about some form of journalling - LVM or btrfs - for your root file-system to protect against a messed up upgrade - take a snapshot, upgrade, and if anything goes wrong it's an easy roll-back. Likewise, do the same for the rotating rust, and use that to back up /home - you can use some option to rsync that only over-writes what's changed, so you do a "snapshot then back up" and have loads of backups going back however far ... Cheers, Wol
Re: [gentoo-user] SDD strategies...
On 17/03/2020 11:54, madscientistatlarge wrote: The issue is not usually end of trusted life, but rather random failure. I've barely managed to recover failed hard drives, That is less likely on SSD though possibly less likely to happen. The drive may be less likely to fail, but I'd say raid or backups are a necessity. From what I've heard, SSDs tend to go read-only when they fail (that's fine), BUT SELF-DESTRUCT ON A POWER CYCLE!!! So don't bank on being able to access a failed SSD after a reboot. Cheers, Wol
Re: [gentoo-user] SDD strategies...
james wrote: > On 3/17/20 10:14 AM, Rich Freeman wrote: >> On Tue, Mar 17, 2020 at 1:59 AM wrote: > >> Finally, ALL DRIVES FAIL. It doesn't matter what the underlying >> storage technology is. I've seen hard drives fail in less than a >> year, with the warranty replacement drive failing less than a year >> after that. I think next warranty replacement (still in the original >> warranty period) lasted 5+ years of near-continuous use. The typical >> failure modes of hard drives and solid state storage are different, >> but they all fail. You can't perfectly predict WHEN they will fail >> either. Most drives have SMART and sometimes it can detect failure >> conditions before failure, but not always. > > > Hello Rich, et al. > > I have deleted most, because I agree with the thread details, you get > what you pay for, but excess payment is rarely rewarded... > > > HEAT is the enemy of all electronics and mechanical things, computer > drives/memory are no exception. There are a myriad of interfaces/codes > on modern motherboards, and quite a few on legacy motherboards that > track heat. Some are not very accurate, but most, are reasonable. > > Hopefully, you kept your mobo book. A section somewhere talks about > temperature sensors. If the cpu is loaded, the drives are most likely > getting hot. If the fans are running on a relatively high speed, the > system is generating tons of heat. If the GPU(s) are running ho9t, the > drives are hot. tools that scan the hardware for sensors are great, > use them! > > > I now install 'water coolers' from thermaltake on all my chassis based > system. new or large video cards have tons of processing going on > inside the GPUs; thus a large source of heat. Systems with lots of > GPU cards, are like ovens. All of this heat, regardless of source, > KILLS all forms of memory, especially 'drives'. Keep everything > monitored, well vented and in a room, cool as possible. Many server > farm rooms run below 50 degrees F, to extend the performance and life > of electronics, particularly HDD and other forms of memory. Many > chipsets, scale down, upon increased heat, auto-magically. > > > Another (indirect) way to monitor heat, is to monitor the power > consumption of a component. (relatively) large power draw, is entwined > with heat production. Heat kills drives and memory no exceptions! > > > Here are few one-liners I use to monitor > (use/load==heat): > > watch -n12 sensors -f > > dstat -tcndylp --top-cpu 10 > > htop > > What would be great, is if folks just list what they use to monitor > the workload (and therefor heat indirectly) or the actual temperatures > of given chipsets and "smart drives"? Perhaps we can then cull the > responses and update of the gentoo help pages online with more > detailed examples, scripts and tools to better organize heat, current > and other relative performance parameters. > > > hth, > James > > I agree that heat is a huge problem. Run a CPU without a heat sink or with a poor one for a while and see how badly that ends. :/ That is why I build my own rigs. I have a Cooler Master HAF-932 case with those large 230MM fans. It has one on the front, one on the side and one on the top. They don't spin fast but they move a lot of air. The hard drives are mounted right behind the front fan which gives them good air flow. When I build, I never use the stock CPU cooler. I shop around and get a large cooler with a large fan. I ended up with the ZALMAN CNPS10X. At the time, it was among the top coolers. I've since replaced the 120MM fan. Using a IR sensor, which is pretty accurate, everything on my computer runs cool. Even my video card runs cool compared to some I've seen posted. My memory comes with coolers as well. They tend to run cool since that side fan blows air right on them but the heat sinks make them run even cooler. The other issue I've read about, start up. When things start up, like hard drives, they pull additional power, especially things like HDD motors. Those put stress on the motor itself plus the controllers that drive them. It also makes the power supply work harder trying to stabilize power which can cause spikes, sags etc. That's not good for the power supply or all the things connected to it. Although, a good power supply deals with that fairly well nowadays. Of course a poorly designed power supply doesn't. Of course, another killer, surges and bad power. We all know those can be bad, either immediately or as life shorteners when minor. Lightening strikes even far away can cause issues. We won't even go into those strikes that are close by and explode light bulbs and such. You know, those that even a surge protector has trouble stopping. I mostly use gkrellm to monitor my temps and such. The temps I get from IR sensors are pretty close to what I see with gkrellm. Since I've never had one, do SSDs get warm or hot? Or do they run cool bare? Curious because I got some options to mount one in
Re: [gentoo-user] SDD strategies...
On 18/3/20 7:25 am, james wrote: > On 3/17/20 10:14 AM, Rich Freeman wrote: >> On Tue, Mar 17, 2020 at 1:59 AM wrote: > >> Finally, ALL DRIVES FAIL. It doesn't matter what the underlying >> storage technology is. I've seen hard drives fail in less than a I gave up trying to do fancy write minimization strategies for SSD's a long time ago as they usually had a performance penalty and I am using SSD's for their considerable speedup - I currently have about 5 SSD's and in addition you can add laptop nvme, m2.nvme etc to the list as well. I run normally - that is swap and compiling caches etc on the SSD. Over that last few years I have had one SSD fail (and 3-4 spinning rust!) - the SSD failure was a random event (it was a "good" intel one) as it just died out of the blue. It was being used as a bcache cacheing drive at the time, and one of the 4 HDD's in the system failed around the same time so I suspect and external event rather than internal to the SSD. The OS drives with swap and compiling have not caused any problems - one early generation 60GB spent its first 18months as a ceph node (really hammers the drive) and its still the main OS drive on my desktop. My comment is that these days, SSD's are not a concern or warrant special treatment and that an SSD failure is likely to be sudden and catastrophic unlike a normal HDD which usually degrades and gives warning signs of impending doom :) BillK
Re: [gentoo-user] SDD strategies...
On 3/17/20 10:14 AM, Rich Freeman wrote: On Tue, Mar 17, 2020 at 1:59 AM wrote: Finally, ALL DRIVES FAIL. It doesn't matter what the underlying storage technology is. I've seen hard drives fail in less than a year, with the warranty replacement drive failing less than a year after that. I think next warranty replacement (still in the original warranty period) lasted 5+ years of near-continuous use. The typical failure modes of hard drives and solid state storage are different, but they all fail. You can't perfectly predict WHEN they will fail either. Most drives have SMART and sometimes it can detect failure conditions before failure, but not always. Hello Rich, et al. I have deleted most, because I agree with the thread details, you get what you pay for, but excess payment is rarely rewarded... HEAT is the enemy of all electronics and mechanical things, computer drives/memory are no exception. There are a myriad of interfaces/codes on modern motherboards, and quite a few on legacy motherboards that track heat. Some are not very accurate, but most, are reasonable. Hopefully, you kept your mobo book. A section somewhere talks about temperature sensors. If the cpu is loaded, the drives are most likely getting hot. If the fans are running on a relatively high speed, the system is generating tons of heat. If the GPU(s) are running ho9t, the drives are hot. tools that scan the hardware for sensors are great, use them! I now install 'water coolers' from thermaltake on all my chassis based system. new or large video cards have tons of processing going on inside the GPUs; thus a large source of heat. Systems with lots of GPU cards, are like ovens. All of this heat, regardless of source, KILLS all forms of memory, especially 'drives'. Keep everything monitored, well vented and in a room, cool as possible. Many server farm rooms run below 50 degrees F, to extend the performance and life of electronics, particularly HDD and other forms of memory. Many chipsets, scale down, upon increased heat, auto-magically. Another (indirect) way to monitor heat, is to monitor the power consumption of a component. (relatively) large power draw, is entwined with heat production. Heat kills drives and memory no exceptions! Here are few one-liners I use to monitor (use/load==heat): watch -n12 sensors -f dstat -tcndylp --top-cpu 10 htop What would be great, is if folks just list what they use to monitor the workload (and therefor heat indirectly) or the actual temperatures of given chipsets and "smart drives"? Perhaps we can then cull the responses and update of the gentoo help pages online with more detailed examples, scripts and tools to better organize heat, current and other relative performance parameters. hth, James
Re: [gentoo-user] SDD strategies...
Hello, On Tue, 17 Mar 2020, Neil Bothwick wrote: >On Tue, 17 Mar 2020 10:43:58 +0100, Andrea Conti wrote: >> >> The SSD is currently reporting 98% of its rated life left: I feel >> >> quite confident it's going to outlast the laptop's useful life. >> > What are you using to get that niformation? >> >> smartctl -A /dev/sdX > >83% after five years of recompiling LO and Chromium, not bad :) # smartctl -a /dev/sda Device Model: SAMSUNG SSD 830 Series User Capacity:128,035,676,160 bytes [128 GB] Sector Size: 512 bytes logical/physical [..] 9 Power_On_Hours 0x0032 091 091 000Old_age Always - 42855 177 Wear_Leveling_Count 0x0013 093 093 000Pre-fail Always - 245 241 Total_LBAs_Written 0x0032 099 099 000Old_age Always - 12835630376 [9 would be ~4.9 years nonstop, 241 is almost 6TiB written, or about 50ish full drive writes] That's been running here since Jul 2012, but I tried to reduce unnecessary writes to it. I did use it for PORTAGE_TMPDIR for a while but that rather quickly reduces the wear leveling from 96 to the current 93. HTH, -dnh -- The Royal Architects were tasked with constructing an edifice known as the Organisation's In[ft]ernal Webshite. It was grand, it was shiney, it was complex, and its navigability would seem to indicate the chief architect was the bastard child of Escher and Giger.-- Niklas Karlsson
Re: [gentoo-user] SDD strategies...
On Tue, Mar 17, 2020 at 1:59 AM wrote: > > The HD will contain the whole system including the complete root > filesustem. Updateing, installing via Gentoo tools will run using > the HD. If that process has ended, I will rsync the HD based root > fileystem to the SSD. > ... I'll go ahead and write one consolidated reply to a couple of points raised in these two threads to save everybody emails. First, I'll echo what was said about this being probably an overly-complex solution to the problem. I think you'll spend more time dealing with this than with any SSD failure issues. Next, in general you tend to get what you pay for. With SSDs if you're getting that bargain 512GB drive for $15 when all the comparable drives are $70, and it was made by some company you've never heard of, chances are that you're missing something. I'm not saying you need to go buy that 150% more expensive "signature edition" drive or whatever, though it might very well have some feature that justifies its price. Just be wary if things that look too good to be true. If you buy a reputable brand that is marketed for your intended use chances are you're getting something decent, even if you're paying 10% more. If you really know what you're doing you can certainly use research to save money by critically evaluating your options. Finally, ALL DRIVES FAIL. It doesn't matter what the underlying storage technology is. I've seen hard drives fail in less than a year, with the warranty replacement drive failing less than a year after that. I think next warranty replacement (still in the original warranty period) lasted 5+ years of near-continuous use. The typical failure modes of hard drives and solid state storage are different, but they all fail. You can't perfectly predict WHEN they will fail either. Most drives have SMART and sometimes it can detect failure conditions before failure, but not always. What will you do when your brand new drive fails 3 weeks after you buy it? If you don't have an answer that doesn't involve you losing stuff you don't want to lose, or having downtime you don't want to have, then you need to re-evaluate your approach. Backups and RAID are the obvious solutions - with backup generally being the more reliable solution to data loss, and RAID being the more reliable solution to downtime, but with them both having some overlap. Here is what I've done: 1. Preferred solution to SSD failure and associated downtime is RAID+backups. The most important stuff is backed up to the cloud, encrypted. With SSDs I usually do a full backup to hard drives since that is fairly inexpensive given their relative capacities. I'm using ZFS mirroring as my RAID-like solution right now, and I use ZFS-send/receive to do hourly backups that are very low overhead. 2. If I'm too cheap to use RAID on a host then I just do the hourly ZFS remote snapshots - that is a good solution on hosts where downtime doesn't matter, because I can just get a new drive and restore the snapshot and I'm back in business after a day or two, accepting a one hour recovery point objective. You can of course use rsync as well. For rsync-based backups I recommend rsnapshot, in portage. For zfs remote snapshots I have switched to zfs_autobackup: https://github.com/psy0rz/zfs_autobackup/blob/master/README.md -- Rich
Re: [gentoo-user] SDD strategies...
Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On Tuesday, March 17, 2020 5:18 AM, Peter Humphrey wrote: > > Hm. My NVMe boot drive doesn't show a lifetime attribute, but my two 1TB SSDs > do, and they both show 100%, which makes me suspicious that either the data > are not being collected or they're being misinterpreted. > > - > > Regards, > Peter. At best that would be how much rewriting the cells still have left. Just as in hard drives there is no way of knowing when a random failure, due to a broken bond wire or a bad solder joint will occur. "Life time" can be calculated for hard drives as well, but that doesn't really mean it won't crap out tomorrow after claiming to have years left. If you really care about your' data, Raid (preferably raid 6 or better) and backups (preferably off site) are the only way to go. The issue is not usually end of trusted life, but rather random failure. I've barely managed to recover failed hard drives, That is less likely on SSD though possibly less likely to happen.
Re: [gentoo-user] SDD strategies...
On Tuesday, 17 March 2020 09:43:58 GMT Andrea Conti wrote: > On 17/03/20 10:03, Neil Bothwick wrote: > > On Tue, 17 Mar 2020 09:35:10 +0100, Andrea Conti wrote: > >> The SSD is currently reporting 98% of its rated life left: I feel quite > >> confident it's going to outlast the laptop's useful life. > > > > What are you using to get that niformation? > > smartctl -A /dev/sdX > > All SSDs I have expose a "life left" attribute, altough the specific > name and encoding of the value will vary. Hm. My NVMe boot drive doesn't show a lifetime attribute, but my two 1TB SSDs do, and they both show 100%, which makes me suspicious that either the data are not being collected or they're being misinterpreted. -- Regards, Peter.
Re: [gentoo-user] SDD strategies...
On Tuesday, 17 March 2020 08:35:10 GMT Andrea Conti wrote: > Hello, > > > SSDs are a common replacement for HDs nowaday -- but I still trust my > > HDs more than this "flashy" things...call me retro or oldschool, but > > it my current "Bauchgefühl" (gut feeling). > > The days of shitty JMicron stuff and OCZ drives dropping like flies are > long gone... O_O I have gentoo booting and running off an OCZ drive since the beginning of 2014. Emerges are on /tmp and /var/ is on a spinning disk. Everything is still running (keeps fingers crossed). I recall a statement from a guy who works in a data centre with 10s of thousands of drives, that SSDs have become more reliable than spinning disks. His replacement rate for spinning drives was significantly higher and their life span shorter. In his vew the business case for moving the whole farm to SSD was clear from a TCO perspective, not withstanding read/write access benefits. I should note I have left some space unallocated on the OCZ for wear levelling reasons - this may have helped in its longevity. In any case, regular back ups of the SSD will come in their own if the drive dies before I replace the PC. signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] SDD strategies...
Neil Bothwick wrote: > On Tue, 17 Mar 2020 10:43:58 +0100, Andrea Conti wrote: > The SSD is currently reporting 98% of its rated life left: I feel quite confident it's going to outlast the laptop's useful life. >>> What are you using to get that niformation? >> smartctl -A /dev/sdX > 83% after five years of recompiling LO and Chromium, not bad :) > > Dang. That makes me feel better about getting one now. Heck, I'm using a 160GB drive currently for my OS. Even a 240GB SDD isn't bad nowadays. I could even make /boot much larger and install a rescue system on there, I think. Since sysrescue went sideways, not sure how that work anymore. I read sysrescue went to some other OS, which should work but you know. ;-) I think this is going to be neato. Now to save up and get one. O_O Dale :-) :-)
Re: [gentoo-user] SDD strategies...
On Tue, Mar 17, 2020 at 09:15:58AM +, Peter Humphrey wrote: > On Tuesday, 17 March 2020 09:04:55 GMT Petr Vaněk wrote: > > > I use tmpfs to reduce compilation writes [1]. > > > > tmpfs /var/tmp/portage/ > > tmpfs uid=portage,gid=portage,mode=0775,size=2G,noatime 0 0 > > tmpfs /tmp/ tmpfs > mode=0777,size=1G,noexec,nosuid,noatime 0 0 > > > > 2G is usually enough for most of packages. > > > > [1] https://wiki.gentoo.org/wiki/Portage_TMPDIR_on_tmpfs > > Do you need to specify a size? I just let the kernel juggle the allocation of > memory to tmpfs and everything else. That is what it's for, no? :) No, you don't but then the system sets size to 50% of physical memory (see man 5 tmpfs). It is an upper bound, just in case if something strange happens.
Re: [gentoo-user] SDD strategies...
On Tue, 17 Mar 2020 10:43:58 +0100, Andrea Conti wrote: > >> The SSD is currently reporting 98% of its rated life left: I feel > >> quite confident it's going to outlast the laptop's useful life. > > What are you using to get that niformation? > > smartctl -A /dev/sdX 83% after five years of recompiling LO and Chromium, not bad :) -- Neil Bothwick A wok is what you throw at a wabbit. pgpsh_REIvFtd.pgp Description: OpenPGP digital signature
Re: [gentoo-user] SDD strategies...
On 17/03/20 10:03, Neil Bothwick wrote: On Tue, 17 Mar 2020 09:35:10 +0100, Andrea Conti wrote: The SSD is currently reporting 98% of its rated life left: I feel quite confident it's going to outlast the laptop's useful life. What are you using to get that niformation? smartctl -A /dev/sdX All SSDs I have expose a "life left" attribute, altough the specific name and encoding of the value will vary. andrea
Re: [gentoo-user] SDD strategies...
On Tuesday, 17 March 2020 09:04:55 GMT Petr Vaněk wrote: > I use tmpfs to reduce compilation writes [1]. > > tmpfs /var/tmp/portage/ > tmpfs uid=portage,gid=portage,mode=0775,size=2G,noatime 0 0 > tmpfs /tmp/ tmpfs mode=0777,size=1G,noexec,nosuid,noatime 0 0 > > 2G is usually enough for most of packages. > > [1] https://wiki.gentoo.org/wiki/Portage_TMPDIR_on_tmpfs Do you need to specify a size? I just let the kernel juggle the allocation of memory to tmpfs and everything else. That is what it's for, no? :) -- Regards, Peter.
Re: [gentoo-user] SDD strategies...
On Tue, Mar 17, 2020 at 06:59:53AM +0100, tu...@posteo.de wrote: > Hi, > > currentlu I am setting up a new PC for my 12-years old one, > which has reached the limits of its "computational power" :) > > SSDs are a common replacement for HDs nowaday -- but I still trust my > HDs more than this "flashy" things...call me retro or oldschool, but > it my current "Bauchgefühl" (gut feeling). > > To reduce write cycles to the SSD, which are quite a lot when using > UNIX/Limux (logging etc) and especially GENTOO (compiling sources > instead of using binary packages -- which is GOOD!), I am planning > the following setup: > > The sustem will boot from SSD. > > The HD will contain the whole system including the complete root > filesustem. Updateing, installing via Gentoo tools will run using > the HD. If that process has ended, I will rsync the HD based root > fileystem to the SSD. > > Folders, which will be written to by the sustem while running will > be symlinked to the HD. > > This should work...? > > Or is there another idea to setup a system which will benefit from > the advantages of a SSD by avoiding its disadvantages? I use tmpfs to reduce compilation writes [1]. tmpfs /var/tmp/portage/ tmpfs uid=portage,gid=portage,mode=0775,size=2G,noatime 0 0 tmpfs /tmp/ tmpfs mode=0777,size=1G,noexec,nosuid,noatime 0 0 2G is usually enough for most of packages. [1] https://wiki.gentoo.org/wiki/Portage_TMPDIR_on_tmpfs Petr
Re: [gentoo-user] SDD strategies...
On Tue, 17 Mar 2020 09:35:10 +0100, Andrea Conti wrote: > The SSD is currently reporting 98% of its rated life left: I feel quite > confident it's going to outlast the laptop's useful life. What are you using to get that niformation? > That's not a single datapoint; every system I have around has used an > SSD as a primary disk for years now, and I've yet to see one fail or > develop any kind of corruption issue. In the same timespan I've had a > fair number of HDD failures. Same here. The main advantage of spinning HDs are that they are cheaper to replace when they fail. I only use them when I need lots of space. -- Neil Bothwick God is real, unless specifically declared integer. pgpYy0LZ8QeUR.pgp Description: OpenPGP digital signature
Re: [gentoo-user] SDD strategies...
Hello, SSDs are a common replacement for HDs nowaday -- but I still trust my HDs more than this "flashy" things...call me retro or oldschool, but it my current "Bauchgefühl" (gut feeling). The days of shitty JMicron stuff and OCZ drives dropping like flies are long gone... you are not going to encounter write endurance problems with a modern SSD from a reputable brand and any kind of reasonable workload. Stay clear from QLC drives and you'll be fine. I have a laptop with a 256GB Plextor M5M SSD installed in 2014. I dual boot Gentoo and Windows, and in addition to the normal stuff, on the Gentoo side I do a couple of world updates per week -- which with a full KDE desktop involves quite a bit of compiling and writing around. The SSD is currently reporting 98% of its rated life left: I feel quite confident it's going to outlast the laptop's useful life. That's not a single datapoint; every system I have around has used an SSD as a primary disk for years now, and I've yet to see one fail or develop any kind of corruption issue. In the same timespan I've had a fair number of HDD failures. The HD will contain the whole system including the complete root filesustem. Updateing, installing via Gentoo tools will run using the HD. If that process has ended, I will rsync the HD based root fileystem to the SSD. > Folders, which will be written to by the sustem while running will be symlinked to the HD. This should work...? It will probably work, if you hack at it long enough :D But seriously, what's the point? Setting up a patchwork of a filesystem like that and maintaining it in time is going to be a complexity and reliability nightmare: if you're going to those lengths because you don't trust SSDs, why have an SSD at all? andrea
Re: [gentoo-user] SDD strategies...
tu...@posteo.de wrote: > Hi, > > currentlu I am setting up a new PC for my 12-years old one, > which has reached the limits of its "computational power" :) > > SSDs are a common replacement for HDs nowaday -- but I still trust my > HDs more than this "flashy" things...call me retro or oldschool, but > it my current "Bauchgefühl" (gut feeling). > > To reduce write cycles to the SSD, which are quite a lot when using > UNIX/Limux (logging etc) and especially GENTOO (compiling sources > instead of using binary packages -- which is GOOD!), I am planning > the following setup: > > The sustem will boot from SSD. > > The HD will contain the whole system including the complete root > filesustem. Updateing, installing via Gentoo tools will run using > the HD. If that process has ended, I will rsync the HD based root > fileystem to the SSD. > > Folders, which will be written to by the sustem while running will > be symlinked to the HD. > > This should work...? > > Or is there another idea to setup a system which will benefit from > the advantages of a SSD by avoiding its disadvantages? > > Background: I am normally using a PC a long time and try to avoid > buying things for reasons like being more modern or being newer. > > Any idea to setup such a sustem is heardly welcone -- thank you > very much in advance! > > Cheers! > Meino > I don't have a SDD here but may one day. Here's my thinking. Set up a chroot or a virtual machine thingy on a regular hard drive. Copy your OS to that, do the updates and then copy packages over to your running system. Then you just emerge -k world and all it does is install binaries on the running system using the SSD. The other benefit of this is a much faster update on the running system. Also, if you get part way through a update, qt or KDE for example, you don't end up with a running system with mismatched versions and possibly a system that doesn't function correctly or won't let you do anything at all, or login even. You can fix the build problems in the chroot/VM and use the running system in the meantime. I started doing this recently because I ran into issues where KDE/qt/something else was not completely updated due to failed compiles that stopped the updates. Some programs I needed wouldn't start or no longer would work correctly if already started. It saves me some grief but would keep the larger writes off your SSD and on a HDD. Oh, if the HDD were to fail, no loss there either. Replace it and start over. I have some scripts, that's a VERY generous use of the word, that I use to mount the chroot, copy the updates over and copy the packages over when compiling is done. I'm still perfecting this but so far, it is working nicely and should work for you as well. Someone else may have a even better idea tho. Dale :-) :-)
[gentoo-user] SDD strategies...
Hi, currentlu I am setting up a new PC for my 12-years old one, which has reached the limits of its "computational power" :) SSDs are a common replacement for HDs nowaday -- but I still trust my HDs more than this "flashy" things...call me retro or oldschool, but it my current "Bauchgefühl" (gut feeling). To reduce write cycles to the SSD, which are quite a lot when using UNIX/Limux (logging etc) and especially GENTOO (compiling sources instead of using binary packages -- which is GOOD!), I am planning the following setup: The sustem will boot from SSD. The HD will contain the whole system including the complete root filesustem. Updateing, installing via Gentoo tools will run using the HD. If that process has ended, I will rsync the HD based root fileystem to the SSD. Folders, which will be written to by the sustem while running will be symlinked to the HD. This should work...? Or is there another idea to setup a system which will benefit from the advantages of a SSD by avoiding its disadvantages? Background: I am normally using a PC a long time and try to avoid buying things for reasons like being more modern or being newer. Any idea to setup such a sustem is heardly welcone -- thank you very much in advance! Cheers! Meino
