Re: svn commit: r323923 - in head: lib/libufs sbin/dumpfs sbin/fsck_ffs sbin/fsdb sbin/newfs sys/kern sys/libkern sys/sys sys/ufs/ffs

[Conrad Meyer]

On Fri, Sep 22, 2017 at 5:45 AM, Kirk McKusick  wrote:
> Author: mckusick
> Date: Fri Sep 22 12:45:15 2017
> New Revision: 323923
> URL: https://svnweb.freebsd.org/changeset/base/323923
>
> Log:
>   Continuing efforts to provide hardening of FFS, this change adds a
>   check hash to cylinder groups. If a check hash fails when a cylinder
>   group is read, no further allocations are attempted in that cylinder
>   group until it has been fixed by fsck. This avoids a class of
>   filesystem panics related to corrupted cylinder group maps. The
>   hash is done using crc32c.
>
> ...
>
> Modified: head/sys/libkern/crc32.c
> ==
> --- head/sys/libkern/crc32.cFri Sep 22 12:07:03 2017(r323922)
> +++ head/sys/libkern/crc32.cFri Sep 22 12:45:15 2017(r323923)
> @@ -759,6 +762,7 @@ calculate_crc32c(uint32_t crc32c,
>  const unsigned char *buffer,
>  unsigned int length)
>  {
> +#ifdef _KERNEL
>  #if defined(__amd64__) || defined(__i386__)
> if ((cpu_feature2 & CPUID2_SSE42) != 0) {
> return (sse42_crc32c(crc32c, buffer, length));
> @@ -776,6 +780,7 @@ calculate_crc32c(uint32_t crc32c,
> return (armv8_crc32c(crc32c, buffer, length));
> } else
>  #endif
> +#endif /* _KERNEL */
> if (length < 4) {
> return (singletable_crc32c(crc32c, buffer, length));
> } else {

Userspace x86 ought to be able to use the SSE-enhanced implementation,
too.  It will just need a cpuid check, and can skip the FPU context
manipulation.

Best,
Conrad
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r323923 - in head: lib/libufs sbin/dumpfs sbin/fsck_ffs sbin/fsdb sbin/newfs sys/kern sys/libkern sys/sys sys/ufs/ffs

[Ed Maste]

)On 22 September 2017 at 14:45, Kirk McKusick  wrote:
> Author: mckusick
> Date: Fri Sep 22 12:45:15 2017
> New Revision: 323923
> URL: https://svnweb.freebsd.org/changeset/base/323923
>
> Log:
>   Continuing efforts to provide hardening of FFS, this change adds a

i386 and others are broken after this change (from
https://ci.freebsd.org/job/FreeBSD-head-i386-build/4050/console):

13:25:09 /usr/src/sys/ufs/ffs/ffs_alloc.c:2619:27: error: format
specifies type 'unsigned long' but the argument has type 'uint64_t'
(aka 'unsigned long long') [-Werror,-Wformat]
13:25:09 cg, cgp->cg_ckhash, bp->b_ckhash);
13:25:09 ^~~~
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r323923 - in head: lib/libufs sbin/dumpfs sbin/fsck_ffs sbin/fsdb sbin/newfs sys/kern sys/libkern sys/sys sys/ufs/ffs

[Kirk McKusick]

Author: mckusick
Date: Fri Sep 22 12:45:15 2017
New Revision: 323923
URL: https://svnweb.freebsd.org/changeset/base/323923

Log:
  Continuing efforts to provide hardening of FFS, this change adds a
  check hash to cylinder groups. If a check hash fails when a cylinder
  group is read, no further allocations are attempted in that cylinder
  group until it has been fixed by fsck. This avoids a class of
  filesystem panics related to corrupted cylinder group maps. The
  hash is done using crc32c.
  
  Check hases are added only to UFS2 and not to UFS1 as UFS1 is primarily
  used in embedded systems with small memories and low-powered processors
  which need as light-weight a filesystem as possible.
  
  Specifics of the changes:
  
  sys/sys/buf.h:
  Add BX_FSPRIV to reserve a set of eight b_xflags that may be used
  by individual filesystems for their own purpose. Their specific
  definitions are found in the header files for each filesystem
  that uses them. Also add fields to struct buf as noted below.
  
  sys/kern/vfs_bio.c:
  It is only necessary to compute a check hash for a cylinder
  group when it is actually read from disk. When calling bread,
  you do not know whether the buffer was found in the cache or
  read. So a new flag (GB_CKHASH) and a pointer to a function to
  perform the hash has been added to breadn_flags to say that the
  function should be called to calculate a hash if the data has
  been read. The check hash is placed in b_ckhash and the B_CKHASH
  flag is set to indicate that a read was done and a check hash
  calculated. Though a rather elaborate mechanism, it should
  also work for check hashing other metadata in the future. A
  kernel internal API change was to change breada into a static
  fucntion and add flags and a function pointer to a check-hash
  function.
  
  sys/ufs/ffs/fs.h:
  Add flags for types of check hashes; stored in a new word in the
  superblock. Define corresponding BX_ flags for the different types
  of check hashes. Add a check hash word in the cylinder group.
  
  sys/ufs/ffs/ffs_alloc.c:
  In ffs_getcg do the dance with breadn_flags to get a check hash and
  if one is provided, check it.
  
  sys/ufs/ffs/ffs_vfsops.c:
  Copy across the BX_FFSTYPES flags in background writes.
  Update the check hash when writing out buffers that need them.
  
  sys/ufs/ffs/ffs_snapshot.c:
  Recompute check hash when updating snapshot cylinder groups.
  
  sys/libkern/crc32.c:
  lib/libufs/Makefile:
  lib/libufs/libufs.h:
  lib/libufs/cgroup.c:
  Include libkern/crc32.c in libufs and use it to compute check
  hashes when updating cylinder groups.
  
  Four utilities are affected:
  
  sbin/newfs/mkfs.c:
  Add the check hashes when building the cylinder groups.
  
  sbin/fsck_ffs/fsck.h:
  sbin/fsck_ffs/fsutil.c:
  Verify and update check hashes when checking and writing cylinder groups.
  
  sbin/fsck_ffs/pass5.c:
  Offer to add check hashes to existing filesystems.
  Precompute check hashes when rebuilding cylinder group
  (although this will be done when it is written in fsutil.c
  it is necessary to do it early before comparing with the old
  cylinder group)
  
  sbin/dumpfs/dumpfs.c
  Print out the new check hash flag(s)
  
  sbin/fsdb/Makefile:
  Needs to add libufs now used by pass5.c imported from fsck_ffs.
  
  Reviewed by: kib
  Tested by: Peter Holm (pho)

Modified:
  head/lib/libufs/Makefile
  head/lib/libufs/cgroup.c
  head/lib/libufs/libufs.h
  head/sbin/dumpfs/dumpfs.c
  head/sbin/fsck_ffs/fsutil.c
  head/sbin/fsck_ffs/pass5.c
  head/sbin/fsdb/Makefile
  head/sbin/newfs/mkfs.c
  head/sys/kern/vfs_bio.c
  head/sys/libkern/crc32.c
  head/sys/sys/buf.h
  head/sys/ufs/ffs/ffs_alloc.c
  head/sys/ufs/ffs/ffs_inode.c
  head/sys/ufs/ffs/ffs_snapshot.c
  head/sys/ufs/ffs/ffs_vfsops.c
  head/sys/ufs/ffs/ffs_vnops.c
  head/sys/ufs/ffs/fs.h

Modified: head/lib/libufs/Makefile
==
--- head/lib/libufs/MakefileFri Sep 22 12:07:03 2017(r323922)
+++ head/lib/libufs/MakefileFri Sep 22 12:45:15 2017(r323923)
@@ -5,7 +5,7 @@ LIB=ufs
 SHLIBDIR?= /lib
 SHLIB_MAJOR=   6
 
-SRCS=  block.c cgroup.c inode.c sblock.c type.c ffs_subr.c ffs_tables.c
+SRCS=  block.c cgroup.c crc32.c inode.c sblock.c type.c ffs_subr.c ffs_tables.c
 INCS=  libufs.h
 
 MAN=   bread.3 cgread.3 libufs.3 sbread.3 ufs_disk_close.3
@@ -18,7 +18,7 @@ MLINKS+= ufs_disk_close.3 ufs_disk_fillout.3
 MLINKS+= ufs_disk_close.3 ufs_disk_fillout_blank.3
 MLINKS+= ufs_disk_close.3 ufs_disk_write.3
 
-.PATH:  ${SRCTOP}/sys/ufs/ffs
+.PATH:  ${SRCTOP}/sys/libkern ${SRCTOP}/sys/ufs/ffs
 
 WARNS?=2
 

Modified: head/lib/libufs/cgroup.c
==
--- head/lib/libufs/cgroup.cFri Sep 22 12:07:03 2017