Re: Race to power off harming SATA SSDs

On Mon 2017-05-08 16:40:11, David Woodhouse wrote: > On Mon, 2017-05-08 at 13:50 +0200, Boris Brezillon wrote: > > On Mon, 08 May 2017 11:13:10 +0100 > > David Woodhouse wrote: > > > > > > > > On Mon, 2017-05-08 at 11:09 +0200, Hans de Goede wrote: > > > > > > > > You're

Re: Race to power off harming SATA SSDs

Hello, On Mon, May 08, 2017 at 08:56:15PM +0200, Pavel Machek wrote: > Well... the SMART counter tells us that the device was not shut down > correctly. Do we have reason to believe that it is _not_ telling us > truth? It is more than one device. It also finished power off command successfully.

Re: Race to power off harming SATA SSDs

On Mon 2017-05-08 13:43:03, Tejun Heo wrote: > Hello, > > On Mon, May 08, 2017 at 06:43:22PM +0200, Pavel Machek wrote: > > What I was trying to point out was that storage people try to treat > > SSDs as HDDs... and SSDs are very different. Harddrives mostly survive > > powerfails (with emergency

RE: Race to power off harming SATA SSDs

> Well, you are right.. and I'm responsible. > > What I was trying to point out was that storage people try to treat SSDs as > HDDs... > and SSDs are very different. Harddrives mostly survive powerfails (with > emergency > parking), while it is very, very difficult to make SSD survive random >

Re: Race to power off harming SATA SSDs

Hello, On Mon, May 08, 2017 at 06:43:22PM +0200, Pavel Machek wrote: > What I was trying to point out was that storage people try to treat > SSDs as HDDs... and SSDs are very different. Harddrives mostly survive > powerfails (with emergency parking), while it is very, very difficult > to make SSD

Re: Race to power off harming SATA SSDs

On Mon 2017-05-08 13:50:05, Boris Brezillon wrote: > On Mon, 08 May 2017 11:13:10 +0100 > David Woodhouse wrote: > > > On Mon, 2017-05-08 at 11:09 +0200, Hans de Goede wrote: > > > You're forgetting that the SSD itself (this thread is about SSDs) also has > > > a major

Re: Race to power off harming SATA SSDs

On Mon, 2017-05-08 at 13:50 +0200, Boris Brezillon wrote: > On Mon, 08 May 2017 11:13:10 +0100 > David Woodhouse wrote: > > > > > On Mon, 2017-05-08 at 11:09 +0200, Hans de Goede wrote: > > > > > > You're forgetting that the SSD itself (this thread is about SSDs) also has

Re: Race to power off harming SATA SSDs

Hi! > > 'clean marker' is a good idea... empty pages have plenty of space. > > Well... you lose that space permanently. Although I suppose you could > do things differently and erase a block immediately prior to using it. > But in that case why ever write the cleanmarker? Just maintain a set of

Re: Race to power off harming SATA SSDs

Boris, Am 08.05.2017 um 13:48 schrieb Boris Brezillon: >>> How do you handle the issue during regular write? Always ignore last >>> successfully written block? > > I guess UBIFS can know what was written last, because of the log-based > approach + the seqnum stored along with FS nodes, but I'm

Re: Race to power off harming SATA SSDs

On Mon, 8 May 2017 13:48:07 +0200 Boris Brezillon wrote: > On Mon, 8 May 2017 13:06:17 +0200 > Richard Weinberger wrote: > > > On Mon, May 8, 2017 at 12:49 PM, Pavel Machek wrote: > > > Aha, nice, so it looks

Re: Race to power off harming SATA SSDs

On Mon, 08 May 2017 11:13:10 +0100 David Woodhouse wrote: > On Mon, 2017-05-08 at 11:09 +0200, Hans de Goede wrote: > > You're forgetting that the SSD itself (this thread is about SSDs) also has > > a major software component which is doing housekeeping all the time, so even

Re: Race to power off harming SATA SSDs

On Mon, 2017-05-08 at 12:49 +0200, Pavel Machek wrote: > On Mon 2017-05-08 10:34:08, David Woodhouse wrote: > > > > On Mon, 2017-05-08 at 11:28 +0200, Pavel Machek wrote: > > > > > > > > > Are you sure you have it right in JFFS2? Do you journal block erases? > > > Apparently, that was pretty

Re: Race to power off harming SATA SSDs

On Mon, May 8, 2017 at 12:49 PM, Pavel Machek wrote: > Aha, nice, so it looks like ubifs is a step back here. > > 'clean marker' is a good idea... empty pages have plenty of space. If UBI (not UBIFS) faces an empty block, it also re-erases it. The EC header is uses as clean marker.

Re: Race to power off harming SATA SSDs

On Mon 2017-05-08 10:34:08, David Woodhouse wrote: > On Mon, 2017-05-08 at 11:28 +0200, Pavel Machek wrote: > > > > Are you sure you have it right in JFFS2? Do you journal block erases? > > Apparently, that was pretty much non-issue on older flashes. > > It isn't necessary in JFFS2. It is a

Re: Race to power off harming SATA SSDs

On Mon, 2017-05-08 at 11:06 +0200, Ricard Wanderlof wrote: > > My point is really that say that the problem is in fact not that the erase  > is cut short due to the power fail, but that the software issues a second  > command before the first erase command has completed, for instance, or  > some

Re: Race to power off harming SATA SSDs

On Mon, 2017-05-08 at 11:09 +0200, Hans de Goede wrote: > You're forgetting that the SSD itself (this thread is about SSDs) also has > a major software component which is doing housekeeping all the time, so even > if the main CPU gets reset the SSD's controller may still happily be erasing >

Re: Race to power off harming SATA SSDs

Pavel, On Mon, May 8, 2017 at 11:28 AM, Pavel Machek wrote: > Are you sure you have it right in JFFS2? Do you journal block erases? > Apparently, that was pretty much non-issue on older flashes. This is what the website says, yes. Do you have hardware where you can trigger it? If

Re: Race to power off harming SATA SSDs

On Mon, 2017-05-08 at 11:28 +0200, Pavel Machek wrote: > > Are you sure you have it right in JFFS2? Do you journal block erases? > Apparently, that was pretty much non-issue on older flashes. It isn't necessary in JFFS2. It is a *purely* log-structured file system (which is why it doesn't scale

Re: Race to power off harming SATA SSDs

On Mon 2017-05-08 08:21:34, David Woodhouse wrote: > On Sun, 2017-05-07 at 22:40 +0200, Pavel Machek wrote: > > > > NOTE: unclean SSD power-offs are dangerous and may brick the device in > > > > the worst case, or otherwise harm it (reduce longevity, damage flash > > > > blocks).  It is also not

Re: Race to power off harming SATA SSDs

Hi, On 08-05-17 11:06, Ricard Wanderlof wrote: On Mon, 8 May 2017, David Woodhouse wrote: On Mon, 8 May 2017, David Woodhouse wrote: Our empirical testing trumps your "can never happen" theory :) I'm sure it does. But what is the explanation then? Has anyone analyzed what is going on

Re: Race to power off harming SATA SSDs

On Mon, 8 May 2017, David Woodhouse wrote: > > On Mon, 8 May 2017, David Woodhouse wrote: > > > Our empirical testing trumps your "can never happen" theory :) > > > > I'm sure it does. But what is the explanation then? Has anyone analyzed  > > what is going on using an oscilloscope to verify

Re: Race to power off harming SATA SSDs

On Mon, 2017-05-08 at 10:36 +0200, Ricard Wanderlof wrote: > On Mon, 8 May 2017, David Woodhouse wrote: > > Our empirical testing trumps your "can never happen" theory :) > > I'm sure it does. But what is the explanation then? Has anyone analyzed  > what is going on using an oscilloscope to verify

Re: Race to power off harming SATA SSDs

On Mon, 8 May 2017, David Woodhouse wrote: > > I've got a problem with the underlying mechanism. How long does it take to  > > erase a NAND block? A couple of milliseconds. That means that for an erase  > > to be "weak" du to a power fail, the host CPU must issue an erase command,  > > and then

Re: Race to power off harming SATA SSDs

On Mon, 2017-05-08 at 09:38 +0200, Ricard Wanderlof wrote: > On Mon, 8 May 2017, David Woodhouse wrote: > > > > > > > > > [Issue is, if you powerdown during erase, you get "weakly erased" > > > page, which will contain expected 0xff's, but you'll get bitflips > > > there quickly. Similar issue

Re: Race to power off harming SATA SSDs

On Mon, 8 May 2017, David Woodhouse wrote: > > [Issue is, if you powerdown during erase, you get "weakly erased" > > page, which will contain expected 0xff's, but you'll get bitflips > > there quickly. Similar issue exists for writes. It is solveable in > > software, just hard and slow... and we

Re: Race to power off harming SATA SSDs

On Sun, 2017-05-07 at 22:40 +0200, Pavel Machek wrote: > > > NOTE: unclean SSD power-offs are dangerous and may brick the device in > > > the worst case, or otherwise harm it (reduce longevity, damage flash > > > blocks).  It is also not impossible to get data corruption. > > > I get that the

Re: Race to power off harming SATA SSDs

Hi! > > However, *IN PRACTICE*, SATA STANDBY IMMEDIATE command completion > > [often?] only indicates that the device is now switching to the target > > power management state, not that it has reached the target state. Any > > further device status inquires would return that it is in STANDBY

Re: Race to power off harming SATA SSDs

Am Dienstag, 11. April 2017, 11:31:29 CEST schrieb Henrique de Moraes Holschuh: > On Tue, 11 Apr 2017, Martin Steigerwald wrote: > > I do have a Crucial M500 and I do have an increase of that counter: > > > > martin@merkaba:~[…]/Crucial-M500> grep "^174" smartctl-a-201* > >

Re: Race to power off harming SATA SSDs

On Tue, 11 Apr 2017, Martin Steigerwald wrote: > I do have a Crucial M500 and I do have an increase of that counter: > > martin@merkaba:~[…]/Crucial-M500> grep "^174" smartctl-a-201* > smartctl-a-2014-03-05.txt:174 Unexpect_Power_Loss_Ct 0x0032 100 100 > 000 > Old_age Always

Re: Race to power off harming SATA SSDs

Am Dienstag, 11. April 2017, 08:52:06 CEST schrieb Tejun Heo: > > Evidently, how often the SSD will lose the race depends on a platform > > and SSD combination, and also on how often the system is powered off. > > A sluggish firmware that takes its time to cut power can save the day... > > > > >

Re: Race to power off harming SATA SSDs

On Mon, 10 Apr 2017, James Bottomley wrote: > On Tue, 2017-04-11 at 08:52 +0900, Tejun Heo wrote: > [...] > > > Any comments? Any clues on how to make the delay "smarter" to > > > trigger only once during platform shutdown, but still trigger per > > > -device when doing per-device hotswapping ?

Re: Race to power off harming SATA SSDs

On Tue, 11 Apr 2017, Tejun Heo wrote: > > The kernel then continues the shutdown path while the SSD is still > > preparing itself to be powered off, and it becomes a race. When the > > kernel + firmware wins, platform power is cut before the SSD has > > finished (i.e. the SSD is subject to an

Re: Race to power off harming SATA SSDs

On Tue, 2017-04-11 at 08:52 +0900, Tejun Heo wrote: [...] > > Any comments? Any clues on how to make the delay "smarter" to > > trigger only once during platform shutdown, but still trigger per > > -device when doing per-device hotswapping ? > > So, if this is actually an issue, sure, we can

Re: Race to power off harming SATA SSDs

Hello, On Mon, Apr 10, 2017 at 08:21:19PM -0300, Henrique de Moraes Holschuh wrote: ... > Per spec (and device manuals), SCSI, SATA and ATA-attached SSDs must be > informed of an imminent poweroff to checkpoing background tasks, flush > RAM caches and close logs. For SCSI SSDs, you must tissue a

Re: Race to power off harming SATA SSDs

On Mon, 10 Apr 2017, Bart Van Assche wrote: > On Mon, 2017-04-10 at 20:21 -0300, Henrique de Moraes Holschuh wrote: > > A proof of concept patch is attached > > Thank you for the very detailed write-up. Sorry but no patch was attached > to the e-mail I received from you ... Indeed. It should

Race to power off harming SATA SSDs

Summary: Linux properly issues the SSD prepare-to-poweroff command to SATA SSDs, but it does not wait for long enough to ensure the SSD has carried it through. This causes a race between the platform power-off path, and the SSD device. When the SSD loses the race, its power is cut while it is