Re: Properly sync filesystems during {suspend,hibernate}/resume

2017-12-20 Thread Grégoire Jadi
"Theo de Raadt"  writes:

>> > I'd appreciate reports about how well it delivers and if any new
>> > problems show up.
>> 
>> Tested on amd64 with softdep and softraid (CRYPTO).
>> Works as expected with suspend and hibernate.
>
> Thanks.
>
>> However, in both cases there is a warning in dmesg during the reboot :
>> > softraid0 sd1 was not shutdown properly
>
> But that is not new.  Take an older kernel, and perform the test of
> suspend followed by removing the power.
>
> Previous to this diff there was no code to gaurantee filesystem sync,
> and no code to syncronize softraid.  I've written the filesystem sync
> code, but I haven't written any softraid syncronization code.

Ok, sorry for the false positive.

Best.



Re: Properly sync filesystems during {suspend,hibernate}/resume

2017-12-19 Thread Theo de Raadt
> > I'd appreciate reports about how well it delivers and if any new
> > problems show up.
> 
> Tested on amd64 with softdep and softraid (CRYPTO).
> Works as expected with suspend and hibernate.

Thanks.

> However, in both cases there is a warning in dmesg during the reboot :
> > softraid0 sd1 was not shutdown properly

But that is not new.  Take an older kernel, and perform the test of
suspend followed by removing the power.

Previous to this diff there was no code to gaurantee filesystem sync,
and no code to syncronize softraid.  I've written the filesystem sync
code, but I haven't written any softraid syncronization code.



Re: Properly sync filesystems during {suspend,hibernate}/resume

2017-12-19 Thread Matthias Schmidt
Hi,

* Theo de Raadt wrote:
> 
> I'd appreciate reports about how well it delivers and if any new
> problems show up.

I tested the patch on an amd64 Intel NUC running softraid crypto with a
keydisk.  I made a lot of disk activity, suspended the system and pulled
the plug.  Both with and without softdep the file system was marked
clean on the next reboot.

However, I encountered a panic on a reboot.  Acquiring a crash dump does
not worked since ddb got stuck on syncing disks.  I made a couple of
photos and uploaded them to https://kappa.xosc.org/misc/.
Unfortunately, I could not reproduce the crash with several reboots.

Let me know if you need me anything else to test.

Cheers

Matthias


/dev/sd2a on / type ffs (local)
/dev/sd2l on /home type ffs (local, nodev, nosuid, softdep)
/dev/sd2d on /tmp type ffs (local, nodev, nosuid)
/dev/sd2f on /usr type ffs (local, nodev, softdep)
/dev/sd2g on /usr/X11R6 type ffs (local, nodev, softdep)
/dev/sd2h on /usr/local type ffs (local, nodev, wxallowed, softdep)
/dev/sd2k on /usr/obj type ffs (local, noatime, nodev, nosuid, softdep)
/dev/sd2j on /usr/ports type ffs (local, noatime, nodev, nosuid, softdep)
/dev/sd2i on /usr/src type ffs (local, noatime, nodev, nosuid, softdep)
/dev/sd2e on /var type ffs (local, nodev, nosuid, softdep)


OpenBSD 6.2-current (GENERIC.MP) #0: Tue Dec 19 09:42:47 CET 2017
x...@tau.xosc.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 17047859200 (16258MB)
avail mem = 16524275712 (15758MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 3.0 @ 0x7b1d5000 (58 entries)
bios0: vendor Intel Corp. version "SYSKLi35.86A.0062.2017.0831.1905" date 
08/31/2017
bios0: Intel corporation NUC6i5SYB
acpi0 at bios0: rev 2
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP APIC FPDT FIDT MCFG HPET LPIT SSDT SSDT SSDT SSDT DBGP 
DBG2 SSDT SSDT UEFI SSDT DMAR
acpi0: wakeup devices PEGP(S4) PEG0(S4) PEGP(S4) PEG1(S4) PEGP(S4) PEG2(S4) 
SIO1(S3) PXSX(S4) RP09(S4) PXSX(S4) RP10(S4) PXSX(S4) RP11(S4) PXSX(S4) 
RP12(S4) PXSX(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i5-6260U CPU @ 1.80GHz, 1696.55 MHz
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,SGX,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,SENSOR,ARAT
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 23MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4.1.1.1, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Core(TM) i5-6260U CPU @ 1.80GHz, 1696.04 MHz
cpu1: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,SGX,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,SENSOR,ARAT
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 1 (application processor)
cpu2: Intel(R) Core(TM) i5-6260U CPU @ 1.80GHz, 1696.04 MHz
cpu2: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,SGX,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,SENSOR,ARAT
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 1, core 0, package 0
cpu3 at mainbus0: apid 3 (application processor)
cpu3: Intel(R) Core(TM) i5-6260U CPU @ 1.80GHz, 1696.04 MHz
cpu3: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,SGX,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,SENSOR,ARAT
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 1, core 1, package 0
ioapic0 at mainbus0: apid 2 pa 0xfec0, version 20, 120 pins
acpimcfg0 at acpi0 addr 0xe000, bus 0-255
acpihpet0 at acpi0: 2399 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus -1 (PEG0)
acpiprt2 at acpi0: bus -1 (PEG1)
acpiprt3 at acpi0: bus -1 (PEG2)
acpiprt4 at acpi0: bus -1 (

Re: Properly sync filesystems during {suspend,hibernate}/resume

2017-12-19 Thread Grégoire Jadi
Theo de Raadt  writes:

> After recent reboot related improvements, I have some familiarity with
> tricking the vnode layer into a stable state.
>
> This should settle filesystems "clean" onto disk.  If one does a
> suspend and then runs out of battery.. or experiences a crash or out
> of battery during a hibernate, the filesystems will now be in a clean
> state and not require an fsck.
>
> After the screen blanks, there may be 1-2 seconds more IO than before.
> Those weren't hitting the platter, but remained in the vfs softlayer
> through to resume.  The new situation is much better.
>
> I have wanted to solve this problem for years..
>
> There may be problems with softraid again, we'll need to see.  I
> haven't tested softdep but the correct syncronization functions appear
> to be called.  USB sticks get detached during the suspend phase, and
> are as good as before.
>
> This isn't quite finished, and I have a few more tricks up my sleeve,
> and the "vfs_stalling" global will probably become a parameter for
> VFS_SYNC()...
>
> I'd appreciate reports about how well it delivers and if any new
> problems show up.

Tested on amd64 with softdep and softraid (CRYPTO).
Works as expected with suspend and hibernate.

However, in both cases there is a warning in dmesg during the reboot :
> softraid0 sd1 was not shutdown properly

It probably needs 'bioctl -d sd1' or something similar.

Best,

> Index: kern/vfs_subr.c
> ===
> RCS file: /cvs/src/sys/kern/vfs_subr.c,v
> retrieving revision 1.265
> diff -u -p -u -r1.265 vfs_subr.c
> --- kern/vfs_subr.c   14 Dec 2017 20:23:15 -  1.265
> +++ kern/vfs_subr.c   16 Dec 2017 23:50:21 -
> @@ -1583,6 +1583,42 @@ vaccess(enum vtype type, mode_t file_mod
>   return (file_mode & mask) == mask ? 0 : EACCES;
>  }
>
> +int vfs_stalling;
> +
> +int
> +vfs_stallmp(struct mount *mp, struct proc *p, int stall)
> +{
> + int error;
> +
> + if (stall) {
> + error = vfs_busy(mp, VB_WRITE|VB_WAIT);
> + if (error) {
> + printf("%s: busy\n", mp->mnt_stat.f_mntonname);
> + return (error);
> + }
> + uvm_vnp_sync(mp);
> + vfs_stalling = stall;
> + error = VFS_SYNC(mp, MNT_WAIT, p->p_ucred, p);
> + if (error) {
> + printf("%s: failed to sync\n", 
> mp->mnt_stat.f_mntonname);
> + vfs_unbusy(mp);
> + return (error);
> + }
> + } else {
> + vfs_unbusy(mp);
> + }
> + return (0);
> +}
> +
> +void
> +vfs_stall(struct proc *p, int stall)
> +{
> + struct mount *mp, *nmp;
> +
> + TAILQ_FOREACH_REVERSE_SAFE(mp, &mountlist, mntlist, mnt_list, nmp)
> + (void) vfs_stallmp(mp, p, stall);
> +}
> +
>  int
>  vfs_readonly(struct mount *mp, struct proc *p)
>  {
> @@ -1627,10 +1663,8 @@ vfs_rofs(struct proc *p)
>  {
>   struct mount *mp, *nmp;
>
> - TAILQ_FOREACH_REVERSE_SAFE(mp, &mountlist, mntlist, mnt_list, nmp) {
> - /* XXX Here is a race, the next pointer is not locked. */
> + TAILQ_FOREACH_REVERSE_SAFE(mp, &mountlist, mntlist, mnt_list, nmp)
>   (void) vfs_readonly(mp, p);
> - }
>  }
>
>  /*
> Index: ufs/ffs/ffs_vfsops.c
> ===
> RCS file: /cvs/src/sys/ufs/ffs/ffs_vfsops.c,v
> retrieving revision 1.170
> diff -u -p -u -r1.170 ffs_vfsops.c
> --- ufs/ffs/ffs_vfsops.c  14 Dec 2017 20:20:38 -  1.170
> +++ ufs/ffs/ffs_vfsops.c  17 Dec 2017 03:03:06 -
> @@ -1145,7 +1145,8 @@ struct ffs_sync_args {
>  };
>
>  int
> -ffs_sync_vnode(struct vnode *vp, void *arg) {
> +ffs_sync_vnode(struct vnode *vp, void *arg)
> +{
>   struct ffs_sync_args *fsa = arg;
>   struct inode *ip;
>   int error;
> @@ -1193,8 +1194,9 @@ ffs_sync(struct mount *mp, int waitfor, 
>  {
>   struct ufsmount *ump = VFSTOUFS(mp);
>   struct fs *fs;
> - int error, allerror = 0, count;
> + int error, allerror = 0, count, clean, fmod;
>   struct ffs_sync_args fsa;
> + extern int vfs_stalling;
>
>   fs = ump->um_fs;
>   /*
> @@ -1240,12 +1242,23 @@ ffs_sync(struct mount *mp, int waitfor, 
>   VOP_UNLOCK(ump->um_devvp, p);
>   }
>   qsync(mp);
> +
>   /*
>* Write back modified superblock.
>*/
> -
> + clean = fs->fs_clean;
> + fmod = fs->fs_fmod;
> + if (vfs_stalling && (fs->fs_ronly == 0)) {
> + fs->fs_fmod = 1;
> + fs->fs_clean = (fs->fs_flags & FS_UNCLEAN) ? 0 : 1;
> +//   printf("%s clean %d fmod %d\n",
> +//   mp->mnt_stat.f_mntonname, fs->fs_clean,
> +//   fs->fs_fmod);
> + }
>   if (fs->fs_fmod != 0 && (error = ffs_sbupdate(ump, waitfor)) != 0)
>   allerror = error;
> + fs->fs_clean = clean;
> + fs->fs_fmod = fmod;
>
>   return (

Re: Properly sync filesystems during {suspend,hibernate}/resume

2017-12-18 Thread Theo de Raadt
> My ext2 media partition got marked dirty but that's probably
> still expected.

ext2fs_sync syncs the superblocks, but doesn't mark them clean.
I didn't dig deep enough to see what code is missing.



Re: Properly sync filesystems during {suspend,hibernate}/resume

2017-12-18 Thread Lari Rasku
Theo de Raadt kirjoitti 12/17/17 klo 08:44:
> This should settle filesystems "clean" onto disk.  If one does a
> suspend and then runs out of battery.. or experiences a crash or out
> of battery during a hibernate, the filesystems will now be in a clean
> state and not require an fsck.

> I'd appreciate reports about how well it delivers and if any new
> problems show up.

Ran this today on a softdep filesystem on amd64.  No problems in
normal operation, manual syncing, etc.  Tested the durability by
suspending my laptop and yanking out the battery; no fsck on my
FFS root partition next boot, as advertised.  My ext2 media
partition got marked dirty but that's probably still expected.

> Index: kern/vfs_subr.c
> ===
> RCS file: /cvs/src/sys/kern/vfs_subr.c,v
> retrieving revision 1.265
> diff -u -p -u -r1.265 vfs_subr.c
> --- kern/vfs_subr.c   14 Dec 2017 20:23:15 -  1.265
> +++ kern/vfs_subr.c   16 Dec 2017 23:50:21 -
> @@ -1583,6 +1583,42 @@ vaccess(enum vtype type, mode_t file_mod
>   return (file_mode & mask) == mask ? 0 : EACCES;
>  }
>  
> +int vfs_stalling;
> +
> +int
> +vfs_stallmp(struct mount *mp, struct proc *p, int stall)
> +{
> + int error;
> +
> + if (stall) {
> + error = vfs_busy(mp, VB_WRITE|VB_WAIT);
> + if (error) {
> + printf("%s: busy\n", mp->mnt_stat.f_mntonname);
> + return (error);
> + }
> + uvm_vnp_sync(mp);
> + vfs_stalling = stall;
> + error = VFS_SYNC(mp, MNT_WAIT, p->p_ucred, p);
> + if (error) {
> + printf("%s: failed to sync\n", 
> mp->mnt_stat.f_mntonname);
> + vfs_unbusy(mp);
> + return (error);
> + }
> + } else {
> + vfs_unbusy(mp);
> + }
> + return (0);
> +}
> +
> +void
> +vfs_stall(struct proc *p, int stall)
> +{
> + struct mount *mp, *nmp;
> +
> + TAILQ_FOREACH_REVERSE_SAFE(mp, &mountlist, mntlist, mnt_list, nmp)
> + (void) vfs_stallmp(mp, p, stall);
> +}
> +
>  int
>  vfs_readonly(struct mount *mp, struct proc *p)
>  {
> @@ -1627,10 +1663,8 @@ vfs_rofs(struct proc *p)
>  {
>   struct mount *mp, *nmp;
>  
> - TAILQ_FOREACH_REVERSE_SAFE(mp, &mountlist, mntlist, mnt_list, nmp) {
> - /* XXX Here is a race, the next pointer is not locked. */
> + TAILQ_FOREACH_REVERSE_SAFE(mp, &mountlist, mntlist, mnt_list, nmp)
>   (void) vfs_readonly(mp, p);
> - }
>  }
>  
>  /*
> Index: ufs/ffs/ffs_vfsops.c
> ===
> RCS file: /cvs/src/sys/ufs/ffs/ffs_vfsops.c,v
> retrieving revision 1.170
> diff -u -p -u -r1.170 ffs_vfsops.c
> --- ufs/ffs/ffs_vfsops.c  14 Dec 2017 20:20:38 -  1.170
> +++ ufs/ffs/ffs_vfsops.c  17 Dec 2017 03:03:06 -
> @@ -1145,7 +1145,8 @@ struct ffs_sync_args {
>  };
>  
>  int
> -ffs_sync_vnode(struct vnode *vp, void *arg) {
> +ffs_sync_vnode(struct vnode *vp, void *arg)
> +{
>   struct ffs_sync_args *fsa = arg;
>   struct inode *ip;
>   int error;
> @@ -1193,8 +1194,9 @@ ffs_sync(struct mount *mp, int waitfor, 
>  {
>   struct ufsmount *ump = VFSTOUFS(mp);
>   struct fs *fs;
> - int error, allerror = 0, count;
> + int error, allerror = 0, count, clean, fmod;
>   struct ffs_sync_args fsa;
> + extern int vfs_stalling;
>  
>   fs = ump->um_fs;
>   /*
> @@ -1240,12 +1242,23 @@ ffs_sync(struct mount *mp, int waitfor, 
>   VOP_UNLOCK(ump->um_devvp, p);
>   }
>   qsync(mp);
> +
>   /*
>* Write back modified superblock.
>*/
> -
> + clean = fs->fs_clean;
> + fmod = fs->fs_fmod;
> + if (vfs_stalling && (fs->fs_ronly == 0)) {
> + fs->fs_fmod = 1;
> + fs->fs_clean = (fs->fs_flags & FS_UNCLEAN) ? 0 : 1;
> +//   printf("%s clean %d fmod %d\n",
> +//   mp->mnt_stat.f_mntonname, fs->fs_clean,
> +//   fs->fs_fmod);
> + }
>   if (fs->fs_fmod != 0 && (error = ffs_sbupdate(ump, waitfor)) != 0)
>   allerror = error;
> + fs->fs_clean = clean;
> + fs->fs_fmod = fmod;
>  
>   return (allerror);
>  }
> Index: dev/acpi/acpi.c
> ===
> RCS file: /cvs/src/sys/dev/acpi/acpi.c,v
> retrieving revision 1.335
> diff -u -p -u -r1.335 acpi.c
> --- dev/acpi/acpi.c   29 Nov 2017 22:51:01 -  1.335
> +++ dev/acpi/acpi.c   17 Dec 2017 00:05:29 -
> @@ -2439,6 +2439,8 @@ acpi_sleep_state(struct acpi_softc *sc, 
>  {
>   extern int perflevel;
>   extern int lid_action;
> + extern void vfs_stall(struct proc *, int);
> + extern int sys_sync(struct proc *, void *, register_t *);
>   int error = ENXIO;
>   size_t rndbuflen = 0;
>   char *rndbuf = NULL;
> @@ -2482,6 +2484,7 @@ acpi_sleep_state(s

Properly sync filesystems during {suspend,hibernate}/resume

2017-12-16 Thread Theo de Raadt
After recent reboot related improvements, I have some familiarity with
tricking the vnode layer into a stable state.

This should settle filesystems "clean" onto disk.  If one does a
suspend and then runs out of battery.. or experiences a crash or out
of battery during a hibernate, the filesystems will now be in a clean
state and not require an fsck.

After the screen blanks, there may be 1-2 seconds more IO than before.
Those weren't hitting the platter, but remained in the vfs softlayer
through to resume.  The new situation is much better.

I have wanted to solve this problem for years..

There may be problems with softraid again, we'll need to see.  I
haven't tested softdep but the correct syncronization functions appear
to be called.  USB sticks get detached during the suspend phase, and
are as good as before.

This isn't quite finished, and I have a few more tricks up my sleeve,
and the "vfs_stalling" global will probably become a parameter for
VFS_SYNC()...

I'd appreciate reports about how well it delivers and if any new
problems show up.

Index: kern/vfs_subr.c
===
RCS file: /cvs/src/sys/kern/vfs_subr.c,v
retrieving revision 1.265
diff -u -p -u -r1.265 vfs_subr.c
--- kern/vfs_subr.c 14 Dec 2017 20:23:15 -  1.265
+++ kern/vfs_subr.c 16 Dec 2017 23:50:21 -
@@ -1583,6 +1583,42 @@ vaccess(enum vtype type, mode_t file_mod
return (file_mode & mask) == mask ? 0 : EACCES;
 }
 
+int vfs_stalling;
+
+int
+vfs_stallmp(struct mount *mp, struct proc *p, int stall)
+{
+   int error;
+
+   if (stall) {
+   error = vfs_busy(mp, VB_WRITE|VB_WAIT);
+   if (error) {
+   printf("%s: busy\n", mp->mnt_stat.f_mntonname);
+   return (error);
+   }
+   uvm_vnp_sync(mp);
+   vfs_stalling = stall;
+   error = VFS_SYNC(mp, MNT_WAIT, p->p_ucred, p);
+   if (error) {
+   printf("%s: failed to sync\n", 
mp->mnt_stat.f_mntonname);
+   vfs_unbusy(mp);
+   return (error);
+   }
+   } else {
+   vfs_unbusy(mp);
+   }
+   return (0);
+}
+
+void
+vfs_stall(struct proc *p, int stall)
+{
+   struct mount *mp, *nmp;
+
+   TAILQ_FOREACH_REVERSE_SAFE(mp, &mountlist, mntlist, mnt_list, nmp)
+   (void) vfs_stallmp(mp, p, stall);
+}
+
 int
 vfs_readonly(struct mount *mp, struct proc *p)
 {
@@ -1627,10 +1663,8 @@ vfs_rofs(struct proc *p)
 {
struct mount *mp, *nmp;
 
-   TAILQ_FOREACH_REVERSE_SAFE(mp, &mountlist, mntlist, mnt_list, nmp) {
-   /* XXX Here is a race, the next pointer is not locked. */
+   TAILQ_FOREACH_REVERSE_SAFE(mp, &mountlist, mntlist, mnt_list, nmp)
(void) vfs_readonly(mp, p);
-   }
 }
 
 /*
Index: ufs/ffs/ffs_vfsops.c
===
RCS file: /cvs/src/sys/ufs/ffs/ffs_vfsops.c,v
retrieving revision 1.170
diff -u -p -u -r1.170 ffs_vfsops.c
--- ufs/ffs/ffs_vfsops.c14 Dec 2017 20:20:38 -  1.170
+++ ufs/ffs/ffs_vfsops.c17 Dec 2017 03:03:06 -
@@ -1145,7 +1145,8 @@ struct ffs_sync_args {
 };
 
 int
-ffs_sync_vnode(struct vnode *vp, void *arg) {
+ffs_sync_vnode(struct vnode *vp, void *arg)
+{
struct ffs_sync_args *fsa = arg;
struct inode *ip;
int error;
@@ -1193,8 +1194,9 @@ ffs_sync(struct mount *mp, int waitfor, 
 {
struct ufsmount *ump = VFSTOUFS(mp);
struct fs *fs;
-   int error, allerror = 0, count;
+   int error, allerror = 0, count, clean, fmod;
struct ffs_sync_args fsa;
+   extern int vfs_stalling;
 
fs = ump->um_fs;
/*
@@ -1240,12 +1242,23 @@ ffs_sync(struct mount *mp, int waitfor, 
VOP_UNLOCK(ump->um_devvp, p);
}
qsync(mp);
+
/*
 * Write back modified superblock.
 */
-
+   clean = fs->fs_clean;
+   fmod = fs->fs_fmod;
+   if (vfs_stalling && (fs->fs_ronly == 0)) {
+   fs->fs_fmod = 1;
+   fs->fs_clean = (fs->fs_flags & FS_UNCLEAN) ? 0 : 1;
+// printf("%s clean %d fmod %d\n",
+// mp->mnt_stat.f_mntonname, fs->fs_clean,
+// fs->fs_fmod);
+   }
if (fs->fs_fmod != 0 && (error = ffs_sbupdate(ump, waitfor)) != 0)
allerror = error;
+   fs->fs_clean = clean;
+   fs->fs_fmod = fmod;
 
return (allerror);
 }
Index: dev/acpi/acpi.c
===
RCS file: /cvs/src/sys/dev/acpi/acpi.c,v
retrieving revision 1.335
diff -u -p -u -r1.335 acpi.c
--- dev/acpi/acpi.c 29 Nov 2017 22:51:01 -  1.335
+++ dev/acpi/acpi.c 17 Dec 2017 00:05:29 -
@@ -2439,6 +2439,8 @@ acpi_sleep_state(struct acpi_softc *sc, 
 {
extern int perflevel;
extern int lid_ac