Re: Build servers offline due to failed SSD

2021-03-15 Thread Ryan Schmidt
On Mar 11, 2021, at 20:15, Bjarne D Mathiesen wrote: > Have you looked at something like this for fast storage/cache : > > https://eshop.macsales.com/item/OWC/SSDACL4M20GB/ > https://www.amazon.de/ASRock-Ultra-Quad-Controller-Karte-PCI-Express/dp/B079TQ9C6Q/ref=sr_1_6 > > Setting these up in

Re: Using RAM instead of disk for build servers (was: Re: Build servers offline due to failed SSD)

2021-03-15 Thread Ryan Schmidt
On Mar 15, 2021, at 09:50, Steven Smith wrote: > SSD RAID offers speed and fault tolerance. Sure. That either wasn't available or was not within what I was willing to spend in 2016 when I set up this system. > Simple options that are tolerant to a single disk failure are: > > •

Re: Using RAM instead of disk for build servers (was: Re: Build servers offline due to failed SSD)

2021-03-15 Thread Dave Horsfall
On Mon, 15 Mar 2021, Daniel J. Luke wrote: Thanks for including this information - it's similar to experience I've had with SSDs for $work. I'd be really surprised if we care about builds on the xserves in 8-10 years (given our previous experience with the ppc to x86 transition). Somewhat

Re: Using RAM instead of disk for build servers (was: Re: Build servers offline due to failed SSD)

2021-03-15 Thread Daniel J. Luke
On Mar 14, 2021, at 6:38 PM, Ryan Schmidt wrote: > As far as longevity, the previous set of 3 500 GB SSDs I bought for these > servers in 2016 lasted 4-5 years. They were rated for 150 TBW (terabytes > written) and actually endured around 450 TBW by the time they failed, or 3 > times as long

Re: Using RAM instead of disk for build servers (was: Re: Build servers offline due to failed SSD)

2021-03-15 Thread Daniel J. Luke
On Mar 14, 2021, at 6:38 PM, Ryan Schmidt wrote: > As far as longevity, the previous set of 3 500 GB SSDs I bought for these > servers in 2016 lasted 4-5 years. They were rated for 150 TBW (terabytes > written) and actually endured around 450 TBW by the time they failed, or 3 > times as long

Re: Using RAM instead of disk for build servers (was: Re: Build servers offline due to failed SSD)

2021-03-15 Thread Steven Smith
SSD RAID offers speed and fault tolerance. Simple options that are tolerant to a single disk failure are: Free/one extra SSD: Use macOS Disk Utility to RAID 1 together two smaller, inexpensive SSD drives for 100% redundancy. OWC ThunderBay 4 Mini, $279: Use macOS Disk Utility to RAID 1 together

Re: Using RAM instead of disk for build servers (was: Re: Build servers offline due to failed SSD)

2021-03-14 Thread Balthasar Indermuehle
Hi Ryan, thanks for your detailed response. I hadn't thought of some of the build intricacies you mention. Let alone the upcoming silicon change and phasing out of x86. Sounds like your approach is a good balance for longevity, performance, and cost. Cheers Balthasar On Mon, 15 Mar 2021 at

Re: Using RAM instead of disk for build servers (was: Re: Build servers offline due to failed SSD)

2021-03-14 Thread Ryan Schmidt
On Mar 14, 2021, at 06:11, Balthasar Indermuehle wrote: > I used to run mac servers in what now can only be described as the days of > yore... when a 32GB RAM bank cost a lot more than a (spinning) disk - and > those were expensive then too. SSDs were not here yet. I haven't checked > pricing

Re: Build servers offline due to failed SSD

2021-03-14 Thread Balthasar Indermuehle
I used to run mac servers in what now can only be described as the days of yore... when a 32GB RAM bank cost a lot more than a (spinning) disk - and those were expensive then too. SSDs were not here yet. I haven't checked pricing lately, but I'd think you could put 256GB of RAM into a server for

Re: Build servers offline due to failed SSD

2021-03-14 Thread Ryan Schmidt
There was some additional downtime in the last few days but the buildmaster now has a permanent home on a new SSD and is faster than ever. Builds that could not be scheduled during recent downtime have been rescheduled and are in progress. On Mar 14, 2021, at 04:02, Vincent Habchi wrote: >

Re: Build servers offline due to failed SSD

2021-03-14 Thread Vincent Habchi
Hi, Wouldn’t it make sense to use some sort of RAM caching to speed up builds instead of SSD? What’s the point of using a permanent storage device for something that is bound to be erased in a very short time? Maybe I’m way off base, though. V.

Re: Build servers offline due to failed SSD

2021-03-11 Thread Bjarne D Mathiesen
Have you looked at something like this for fast storage/cache : https://eshop.macsales.com/item/OWC/SSDACL4M20GB/ https://www.amazon.de/ASRock-Ultra-Quad-Controller-Karte-PCI-Express/dp/B079TQ9C6Q/ref=sr_1_6 Setting these up in RAID-0 (with proper backup) ought to be the fastest storage solution

Re: Build servers offline due to failed SSD

2021-03-11 Thread Bjarne D Mathiesen
Further discussions : https://arstechnica.com/information-technology/2012/06/inside-the-ssd-revolution-how-solid-state-disks-really-work/ https://arstechnica.com/gadgets/2015/03/consumer-ssds-benchmarked-to-death-and-last-far-longer-than-rated/

Re: Build servers offline due to failed SSD

2021-03-09 Thread Dave C via macports-users
I’m curious. I know the ToH, but “tail”? Dave - - - > ... but I’m going to reconfigure it to get the longer backup “tail” provided > by the ToH approach. > > Jim

Re: Build servers offline due to failed SSD

2021-03-09 Thread James Secan
James, Thanks for the Tower of Hanoi reminder. I used that many (many) years ago with 9" tapes on a Big Iron machine but had forgotten the technique. I’ve been using a FIFO seven-day rotation backup of my main user directory (using CCC), but I’m going to reconfigure it to get the longer

Re: Build servers offline due to failed SSD

2021-03-08 Thread James Linder
> On 9 Mar 2021, at 5:53 am, Dave C via macports-users > wrote: > > Old technology drives use magnetism to hold bits. This works for decades, or > so I’ve read. Usually the motor or bearings die before the magnetic medium > fails. > > Solid State Drives use memory chips to hold bits.

Re: Build servers offline due to failed SSD

2021-03-08 Thread Dave C via macports-users
I think most people who talk about servers and HDs/SSDs are referring to commercial internet-connected servers. Yes, a private server will likely see a lesser degree of service/use and storage drives can be uprated (the opposite of derated) for greater lifetime. Dave > I’ve been looking at

Re: Build servers offline due to failed SSD

2021-03-08 Thread Dave C via macports-users
Old technology drives use magnetism to hold bits. This works for decades, or so I’ve read. Usually the motor or bearings die before the magnetic medium fails. Solid State Drives use memory chips to hold bits. These “bit holders” can wear out after a few trillion transitions (changing from 1

Re: Build servers offline due to failed SSD

2021-03-08 Thread Dave Horsfall
On Sun, 7 Mar 2021, Todd Doucet wrote: HDs fail also, obviously, but tend not to be so predictable about it.  That of course depends upon the HD and the OS; my (FreeBSD) server's drive is around 20 years old, and is still going strong. There's also software that monitors the health of the

Re: Build servers offline due to failed SSD

2021-03-08 Thread Dave Horsfall
On Sun, 7 Mar 2021, Michael A. Leonetti via macports-users wrote: I’d really love to know more about what you’re saying here. Up until I just read what you wrote, I thought SSDs were the savior of HDDs. Real disk drives [tm] have their N/S magnetic poles lined up pretty much forever; SSDs

Re: Build servers offline due to failed SSD

2021-03-08 Thread James Linder
> On 7 Mar 2021, at 3:26 pm, Dave C via macports-users > wrote: > > This applies to affordable SSDs. As you say, the ones that are on par (re. > reliability) with HDDs are $pendy. > > It’s something to do with an SSD’s limited number of write cycles, if I > remember... > > Dave > > - -

Re: Build servers offline due to failed SSD

2021-03-08 Thread Lothar Haeger
Here‘s an in depth discussion on SSD reliability, a little more detailed than „(not) recommended“ from someone with a lot of first hand experience, it seems: https://www.backblaze.com/blog/how-reliable-are-ssds/

Re: Build servers offline due to failed SSD

2021-03-07 Thread Daniel J. Luke
On Mar 7, 2021, at 8:30 PM, Todd Doucet wrote: > I think one can only get so far with purely qualitative analysis of the > characteristics of SSDs and HDs and then the end of that analysis will be > one-size-fits all advice, for example "recommended" or "not recommended" for > servers. this

Re: Build servers offline due to failed SSD

2021-03-07 Thread Peter West
I’ve been looking at VPS providers, and most of them offer SSD-based VPSs, so they seem to be increasingly popular. I suspect that most VPSs do not get consistently hammered, though. Peter — p...@ehealth.id.au “Destroy this temple, and in three days I will raise it up.” > On 8 Mar 2021, at

Re: Build servers offline due to failed SSD

2021-03-07 Thread Todd Doucet
I think one can only get so far with purely qualitative analysis of the characteristics of SSDs and HDs and then the end of that analysis will be one-size-fits all advice, for example "recommended" or "not recommended" for servers. Surely the answer might vary depending on the particular

Re: Build servers offline due to failed SSD

2021-03-07 Thread Dave C via macports-users
To emphasize again, the reason SSDs aren’t recommended for servers is because servers—by definition—see much heavier service, and these read/write cycles are used up more quickly. For personal use in a PC, or such, SSDs are proving to be the dream they were promised to be. As mentioned, given

Re: Build servers offline due to failed SSD

2021-03-07 Thread John Chivian
The “on/off” switches in SSD’s are fragile and essentially break after too many read/write cycles. As pointed out, it’s a get what you pay for world and cheap SSD’s are just that… cheap. The expensive ones are more reliable because they actually make available only a portion of their total

Re: Build servers offline due to failed SSD

2021-03-07 Thread Michael A. Leonetti via macports-users
I’d really love to know more about what you’re saying here. Up until I just read what you wrote, I thought SSDs were the savior of HDDs. Michael A. Leonetti As warm as green tea > 3/7/21 午後5:26、Dave Horsfall のメール: > > On Sat, 6 Mar 2021, Dave C via macports-users wrote: > >> Isn’t SSD a bad

Re: Build servers offline due to failed SSD

2021-03-07 Thread Dave Horsfall
On Sat, 6 Mar 2021, Dave C via macports-users wrote: Isn’t SSD a bad choice for server duty? No server farms use them, apparently due to short lifespan. If you knew how SSDs worked then you wouldn't use them at all without many backups. Give me spinning rust any day... -- Dave

Re: Build servers offline due to failed SSD

2021-03-07 Thread Ryan Schmidt
On Mar 7, 2021, at 00:20, Dave C wrote: > Isn’t SSD a bad choice for server duty? My opinion is that it is a good choice in terms of performance. When I first up this incarnation of our buildbot system in 2016 I had the workers running on SSDs so that builds would be fast (our previous

Re: Build servers offline due to failed SSD

2021-03-06 Thread Dave C via macports-users
This applies to affordable SSDs. As you say, the ones that are on par (re. reliability) with HDDs are $pendy. It’s something to do with an SSD’s limited number of write cycles, if I remember... Dave - - - > Isn’t SSD a bad choice for server duty? No server farms use them, apparently > due

Re: Build servers offline due to failed SSD

2021-03-06 Thread Andrew Udvare
> On 2021-03-07, at 01:20, Dave C via macports-users > wrote: > > Isn’t SSD a bad choice for server duty? No server farms use them, apparently > due to short lifespan. > > Dave Plenty of servers use SSDs now, usually with HDDs to lower cost. The default option on AWS EC2 is to use an SSD.

Re: Build servers offline due to failed SSD

2021-03-06 Thread Dave C via macports-users
Isn’t SSD a bad choice for server duty? No server farms use them, apparently due to short lifespan. Dave

Re: Build servers offline due to failed SSD

2021-03-06 Thread Ryan Schmidt
On Mar 2, 2021, at 09:03, Ryan Schmidt wrote: > On Feb 21, 2021, at 10:08, Ryan Schmidt wrote: > >> We got through the winter storms but now there's a new problem. The SSD that >> the buildmaster VM is stored on and that boots up VMware ESXi is failing. >> I'm currently setting up a new

Re: Build servers offline due to failed SSD

2021-03-02 Thread Ryan Schmidt
On Feb 21, 2021, at 10:08, Ryan Schmidt wrote: > We got through the winter storms but now there's a new problem. The SSD that > the buildmaster VM is stored on and that boots up VMware ESXi is failing. I'm > currently setting up a new ESXi startup disk and trying to find a temporary > disk

Build servers offline due to failed SSD

2021-02-21 Thread Ryan Schmidt
We got through the winter storms but now there's a new problem. The SSD that the buildmaster VM is stored on and that boots up VMware ESXi is failing. I'm currently setting up a new ESXi startup disk and trying to find a temporary disk I can move that VM to to get us back up and running. The