svn commit: r367195 - head

[Colin Percival]

Author: cperciva
Date: Sat Oct 31 03:36:31 2020
New Revision: 367195
URL: https://svnweb.freebsd.org/changeset/base/367195

Log:
  Attempt to defuse a land mine before anyone else steps on it: The
  freebsd-boot partition is not always the first one.
  
  Following the instructions in UPDATING resulted in my overwriting
  the efiboot0 partition on my laptop with ZFS boot blocks, which
  had negative effects on the system's bootability.
  
  Reviewed by:  allanjude
  Differential Revision:https://reviews.freebsd.org/D27002

Modified:
  head/UPDATING

Modified: head/UPDATING
==
--- head/UPDATING   Sat Oct 31 02:57:53 2020(r367194)
+++ head/UPDATING   Sat Oct 31 03:36:31 2020(r367195)
@@ -2203,9 +2203,11 @@ COMMON ITEMS:
 
2.) update the ZFS boot block on your boot drive
 
-   The following example updates the ZFS boot block on the first
-   partition (freebsd-boot) of a GPT partitioned drive ada0:
-   "gpart bootcode -p /boot/gptzfsboot -i 1 ada0"
+   The following example updates the ZFS boot block on the
+   freebsd-boot partition of a GPT partitioned drive ada0:
+   "gpart bootcode -p /boot/gptzfsboot -i $N ada0"
+   The value $N will typically be 1 (if booting from BIOS) or 2 (if
+   booting from EFI).
 
Non-boot pools do not need these updates.
 
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r365898 - head/usr.sbin/freebsd-update

[Colin Percival]

Author: cperciva
Date: Sat Sep 19 02:15:56 2020
New Revision: 365898
URL: https://svnweb.freebsd.org/changeset/base/365898

Log:
  Move finalize_components_config from get_params to cmd_*.
  
  This allows us to redirect its output in cmd_cron, so that the
  "src component not installed, skipped" message will be treated
  the same way as other output from freebsd-update cron: Sent
  in an email to root (or other address specified) if there are
  updates to install, and silenced otherwise.
  
  PR:   202492
  MFC After:1 week
  Differential Revision:https://reviews.freebsd.org/D26432

Modified:
  head/usr.sbin/freebsd-update/freebsd-update.sh

Modified: head/usr.sbin/freebsd-update/freebsd-update.sh
==
--- head/usr.sbin/freebsd-update/freebsd-update.sh  Sat Sep 19 02:12:04 
2020(r365897)
+++ head/usr.sbin/freebsd-update/freebsd-update.sh  Sat Sep 19 02:15:56 
2020(r365898)
@@ -3300,12 +3300,12 @@ get_params () {
parse_cmdline $@
parse_conffile
default_params
-   finalize_components_config ${COMPONENTS}
 }
 
 # Fetch command.  Make sure that we're being called
 # interactively, then run fetch_check_params and fetch_run
 cmd_fetch () {
+   finalize_components_config ${COMPONENTS}
if [ ! -t 0 -a $NOTTYOK -eq 0 ]; then
echo -n "`basename $0` fetch should not "
echo "be run non-interactively."
@@ -3326,6 +3326,7 @@ cmd_cron () {
sleep `jot -r 1 0 3600`
 
TMPFILE=`mktemp /tmp/freebsd-update.XX` || exit 1
+   finalize_components_config ${COMPONENTS} >> ${TMPFILE}
if ! fetch_run >> ${TMPFILE} ||
! grep -q "No updates needed" ${TMPFILE} ||
[ ${VERBOSELEVEL} = "debug" ]; then
@@ -3337,6 +3338,7 @@ cmd_cron () {
 
 # Fetch files for upgrading to a new release.
 cmd_upgrade () {
+   finalize_components_config ${COMPONENTS}
upgrade_check_params
upgrade_run || exit 1
 }
@@ -3344,6 +3346,7 @@ cmd_upgrade () {
 # Check if there are fetched updates ready to install.
 # Chdir into the working directory.
 cmd_updatesready () {
+   finalize_components_config ${COMPONENTS}
# Check if working directory exists (if not, no updates pending)
if ! [ -e "${WORKDIR}" ]; then
echo "No updates are available to install."
@@ -3368,24 +3371,28 @@ cmd_updatesready () {
 
 # Install downloaded updates.
 cmd_install () {
+   finalize_components_config ${COMPONENTS}
install_check_params
install_run || exit 1
 }
 
 # Rollback most recently installed updates.
 cmd_rollback () {
+   finalize_components_config ${COMPONENTS}
rollback_check_params
rollback_run || exit 1
 }
 
 # Compare system against a "known good" index.
 cmd_IDS () {
+   finalize_components_config ${COMPONENTS}
IDS_check_params
IDS_run || exit 1
 }
 
 # Output configuration.
 cmd_showconfig () {
+   finalize_components_config ${COMPONENTS}
for X in ${CONFIGOPTIONS}; do
echo $X=$(eval echo \$${X})
done
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r365696 - head/release/tools

[Colin Percival]

Author: cperciva
Date: Sun Sep 13 19:56:53 2020
New Revision: 365696
URL: https://svnweb.freebsd.org/changeset/base/365696

Log:
  Spawn the DHCPv6 client in EC2 instances via rtsold.
  
  Prior to this commit, EC2 AMIs used a "dual-dhclient" tool which was
  launched in place of dhclient and spawned both the base system dhclient
  for IPv4 and the ISC dhclient from ports for IPv6.
  
  Now that rtsold supports the "M bit" (managed configuration), we can go
  back to having the base system dhclient spawned normally, and provide a
  script to rtsold which spawns the ISC dhclient from ports when rtsold
  decides that it is appropriate.
  
  Thanks to:bz
  MFC after:1 week
  Sponsored by: https://www.patreon.com/cperciva

Modified:
  head/release/tools/ec2.conf

Modified: head/release/tools/ec2.conf
==
--- head/release/tools/ec2.conf Sun Sep 13 19:11:45 2020(r365695)
+++ head/release/tools/ec2.conf Sun Sep 13 19:56:53 2020(r365696)
@@ -6,7 +6,7 @@
 # Packages to install into the image we're creating.  This is a deliberately
 # minimalist set, providing only the packages necessary to bootstrap further
 # package installation as specified via EC2 user-data.
-export VM_EXTRA_PACKAGES="ec2-scripts firstboot-freebsd-update firstboot-pkgs 
dual-dhclient-daemon ebsnvme-id"
+export VM_EXTRA_PACKAGES="ec2-scripts firstboot-freebsd-update firstboot-pkgs 
isc-dhcp44-client ebsnvme-id"
 
 # Include the amazon-ssm-agent package in amd64 images, since some users want
 # to be able to use it on systems which are not connected to the Internet.
@@ -63,9 +63,19 @@ vm_extra_pre_umount() {
# via EC2 user-data.
echo 'firstboot_pkgs_list="awscli"' >> ${DESTDIR}/etc/rc.conf
 
-   # Enable IPv6 on all interfaces, and use DHCP on both IPv4 and IPv6.
+   # Enable IPv6 on all interfaces, and spawn DHCPv6 via rtsold
echo 'ipv6_activate_all_interfaces="YES"' >> ${DESTDIR}/etc/rc.conf
-   echo 'dhclient_program="/usr/local/sbin/dual-dhclient"' >> 
${DESTDIR}/etc/rc.conf
+   echo 'rtsold_enable="YES"' >> ${DESTDIR}/etc/rc.conf
+   echo 'rtsold_flags="-M /usr/local/libexec/rtsold-M -a"' >> 
${DESTDIR}/etc/rc.conf
+
+   # Provide a script which rtsold can use to launch DHCPv6
+   mkdir -p ${DESTDIR}/usr/local/libexec
+   cat > ${DESTDIR}/usr/local/libexec/rtsold-M <<'EOF'
+#!/bin/sh
+
+/usr/local/sbin/dhclient -6 -nw -N -cf /dev/null $1
+EOF
+   chmod 755 ${DESTDIR}/usr/local/libexec/rtsold-M
 
# The EC2 console is output-only, so while printing a backtrace can
# be useful, there's no point dropping into a debugger or waiting
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r365695 - head/release/tools

[Colin Percival]

Author: cperciva
Date: Sun Sep 13 19:11:45 2020
New Revision: 365695
URL: https://svnweb.freebsd.org/changeset/base/365695

Log:
  Bump the size of EC2 AMIs up to 5 GB.
  
  The FreeBSD base system continues to expand.  4GB is now insufficient;
  we passed 3 GB in May 2019; we passed 2 GB in August 2017.  Over half
  of the disk space used is in /usr/lib/debug/.
  
  Without this change, instances boot but are unusable, since the first
  thing which breaks when VM filesystems are too small is the "pkg install"
  in the VM building process.

Modified:
  head/release/tools/ec2.conf

Modified: head/release/tools/ec2.conf
==
--- head/release/tools/ec2.conf Sun Sep 13 18:24:15 2020(r365694)
+++ head/release/tools/ec2.conf Sun Sep 13 19:11:45 2020(r365695)
@@ -19,12 +19,12 @@ fi
 # Set to a list of third-party software to enable in rc.conf(5).
 export VM_RC_LIST="ec2_configinit ec2_fetchkey ec2_loghostkey 
firstboot_freebsd_update firstboot_pkgs ntpd dev_aws_disk"
 
-# Build with a 3.9 GB UFS partition; the growfs rc.d script will expand
+# Build with a 4.9 GB UFS partition; the growfs rc.d script will expand
 # the partition to fill the root disk after the EC2 instance is launched.
 # Note that if this is set to G, we will end up with an  GB disk
 # image since VMSIZE is the size of the UFS partition, not the disk which
 # it resides within.
-export VMSIZE=4000M
+export VMSIZE=5000M
 
 # No swap space; the ec2_ephemeralswap rc.d script will allocate swap
 # space on EC2 ephemeral disks.  (If they exist -- the T2 low-cost instances
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r364832 - head/usr.bin/lockf

[Colin Percival]

Author: cperciva
Date: Wed Aug 26 19:28:30 2020
New Revision: 364832
URL: https://svnweb.freebsd.org/changeset/base/364832

Log:
  Remove whitespace which accidentaly snuck into r364831.

Modified:
  head/usr.bin/lockf/lockf.1

Modified: head/usr.bin/lockf/lockf.1
==
--- head/usr.bin/lockf/lockf.1  Wed Aug 26 19:26:48 2020(r364831)
+++ head/usr.bin/lockf/lockf.1  Wed Aug 26 19:28:30 2020(r364832)
@@ -1,4 +1,4 @@
- .\"
+.\"
 .\" Copyright (C) 1998 John D. Polstra.  All rights reserved.
 .\"
 .\" Redistribution and use in source and binary forms, with or without
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r364831 - head/usr.bin/lockf

[Colin Percival]

Author: cperciva
Date: Wed Aug 26 19:26:48 2020
New Revision: 364831
URL: https://svnweb.freebsd.org/changeset/base/364831

Log:
  Add -w option to lockf(1).
  
  By default, lockf(1) opens its lock file O_RDONLY|O_EXLOCK.  On NFS, if the
  file already exists, this is split into opening the file read-only and then
  requesting an exclusive lock -- and the second step fails because NFS does
  not permit exclusive locking on files which are opened read-only.
  
  The new -w option changes the open flags to O_WRONLY|O_EXLOCK, allowing it
  to work on NFS -- at the cost of not working if the file cannot be opened
  for writing.
  
  (Whether the traditional BSD behaviour of allowing exclusive locks to be
  obtained on a file which cannot be opened for writing is a good idea is
  perhaps questionable since it may allow less-privileged users to perform
  a local denial of service; however this behaviour has been present for a
  long time and changing it now seems like it would cause problems.)
  
  Reviewed by:  rmacklem
  Differential Revision:https://reviews.freebsd.org/D26005

Modified:
  head/usr.bin/lockf/lockf.1
  head/usr.bin/lockf/lockf.c

Modified: head/usr.bin/lockf/lockf.1
==
--- head/usr.bin/lockf/lockf.1  Wed Aug 26 19:03:15 2020(r364830)
+++ head/usr.bin/lockf/lockf.1  Wed Aug 26 19:26:48 2020(r364831)
@@ -1,4 +1,4 @@
-.\"
+ .\"
 .\" Copyright (C) 1998 John D. Polstra.  All rights reserved.
 .\"
 .\" Redistribution and use in source and binary forms, with or without
@@ -24,7 +24,7 @@
 .\"
 .\" $FreeBSD$
 .\"
-.Dd June 18, 2020
+.Dd August 26, 2020
 .Dt LOCKF 1
 .Os
 .Sh NAME
@@ -32,7 +32,7 @@
 .Nd execute a command while holding a file lock
 .Sh SYNOPSIS
 .Nm
-.Op Fl kns
+.Op Fl knsw
 .Op Fl t Ar seconds
 .Ar file
 .Ar command
@@ -121,6 +121,14 @@ When a lock times out,
 is
 .Em not
 executed.
+.It Fl w
+Causes
+.Nm
+to open
+.Ar file
+for writing rather than reading.
+This is necessary on filesystems (including NFSv4) where a file which
+has been opened read-only cannot be exclusively locked.
 .El
 .Pp
 In no event will

Modified: head/usr.bin/lockf/lockf.c
==
--- head/usr.bin/lockf/lockf.c  Wed Aug 26 19:03:15 2020(r364830)
+++ head/usr.bin/lockf/lockf.c  Wed Aug 26 19:26:48 2020(r364831)
@@ -62,9 +62,9 @@ main(int argc, char **argv)
pid_t child;
 
silent = keep = 0;
-   flags = O_CREAT;
+   flags = O_CREAT | O_RDONLY;
waitsec = -1;   /* Infinite. */
-   while ((ch = getopt(argc, argv, "sknt:")) != -1) {
+   while ((ch = getopt(argc, argv, "sknt:w")) != -1) {
switch (ch) {
case 'k':
keep = 1;
@@ -84,6 +84,9 @@ main(int argc, char **argv)
"invalid timeout \"%s\"", optarg);
}
break;
+   case 'w':
+   flags = (flags & ~O_RDONLY) | O_WRONLY;
+   break;
default:
usage();
}
@@ -171,7 +174,7 @@ acquire_lock(const char *name, int flags)
 {
int fd;
 
-   if ((fd = open(name, O_RDONLY|O_EXLOCK|flags, 0666)) == -1) {
+   if ((fd = open(name, O_EXLOCK|flags, 0666)) == -1) {
if (errno == EAGAIN || errno == EINTR)
return (-1);
else if (errno == ENOENT && (flags & O_CREAT) == 0)
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Please include benchmarks in commit messages (was Re: svn commit: r363450 - in head: share/man/man3 sys/sys)

[Colin Percival]

[Picking this as an example of a general trend...]

On 2020-07-23 10:16, Doug Moore wrote:
>   [...]
>   Testing has shown that for the cases where red-black trees do worst,
>   wavl trees better balance leads to faster lookups, so that if lookups
>   outnumber insertions by a nontrivial amount, lookup time saved exceeds
>   the extra cost of balancing.

I'd like to request that commits like this include some benchmarks.  In the
review of these particular changes I see several microbenchmarks for a variety
of use cases; it could be helpful to future code spelunkers if these were
summarized in the commit message (e.g. "lookup-heavy microbenchmarks saw a
X% speedup while write-heavy microbenchmarks saw a Y% slowdown").  It would
also be nice to see a macrobenchmark (e.g. buildworld), even (and perhaps
*especially*) if the result is "no difference was observed".

Again, this is a general request -- this is just the latest in a long series
of commits I've seen by many people which have sounded like they could have
performance impacts but have not included any quantitative results.

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r362791 - head/release/tools

[Colin Percival]

Author: cperciva
Date: Tue Jun 30 06:14:34 2020
New Revision: 362791
URL: https://svnweb.freebsd.org/changeset/base/362791

Log:
  Make EC2 AMIs use portsnap and freebsd-update mirrors hosted in AWS
  
  This adjusts freebsd-update.conf and portsnap.conf files in EC2 AMIs to
  point at the new AWS-hosted mirror network.
  
  Approved by:  re (delphij)
  MFC after:1 month
  Differential Revision:https://reviews.freebsd.org/D25498

Modified:
  head/release/tools/ec2.conf

Modified: head/release/tools/ec2.conf
==
--- head/release/tools/ec2.conf Mon Jun 29 22:12:23 2020(r362790)
+++ head/release/tools/ec2.conf Tue Jun 30 06:14:34 2020(r362791)
@@ -108,6 +108,12 @@ vm_extra_pre_umount() {
's/^#ChallengeResponseAuthentication 
yes/ChallengeResponseAuthentication no/' \
${DESTDIR}/etc/ssh/sshd_config
 
+   # Use FreeBSD Update and Portsnap mirrors hosted in AWS
+   sed -i '' -e 's/update.FreeBSD.org/aws.update.FreeBSD.org/' \
+   ${DESTDIR}/etc/freebsd-update.conf
+   sed -i '' -e 's/portsnap.FreeBSD.org/aws.portsnap.FreeBSD.org/' \
+   ${DESTDIR}/etc/portsnap.conf
+
# Use the NTP service provided by Amazon
sed -i '' -e 's/^pool/#pool/' \
-e '1,/^#server/s/^#server.*/server 169.254.169.123 iburst/' \
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r362557 - head/sbin/dump

[Colin Percival]

Author: cperciva
Date: Tue Jun 23 21:11:40 2020
New Revision: 362557
URL: https://svnweb.freebsd.org/changeset/base/362557

Log:
  Clean up some function and variable names.
  
  The change from "slave" processes to "minion" processes to "worker"
  processes left some less-than-coherent names:
  1. "enslave" turned into the ungrammatical "enworker".
  2. "slp" (SLave Pointer) turned into "mlp" (Minion [L] Pointer?).
  
  Convert "enworker" to "create_workers" (the function in question forks
  off 3 worker processes), and replace "mlp" with "wp" (Worker Pointer)
  and "tmlp" with "twp" (Temporary Worker Pointer).
  
  Reviewed by:  imp, cem, danfe
  Differential Revision:https://reviews.freebsd.org/D25403

Modified:
  head/sbin/dump/tape.c

Modified: head/sbin/dump/tape.c
==
--- head/sbin/dump/tape.c   Tue Jun 23 20:48:43 2020(r362556)
+++ head/sbin/dump/tape.c   Tue Jun 23 21:11:40 2020(r362557)
@@ -76,7 +76,7 @@ staticFILE *popenfp = NULL;
 
 static int atomic(ssize_t (*)(), int, char *, int);
 static void worker(int, int);
-static void enworker(void);
+static void create_workers(void);
 static void flushtape(void);
 static void killall(void);
 static void rollforward(void);
@@ -108,7 +108,7 @@ static struct worker {
char (*tblock)[TP_BSIZE]; /* buffer for data blocks */
struct req *req;/* buffer for requests */
 } workers[WORKERS+1];
-static struct worker *mlp;
+static struct worker *wp;
 
 static char(*nextblock)[TP_BSIZE];
 
@@ -152,11 +152,11 @@ alloctape(void)
(((long)[ntrec + 1] + pgoff) &~ pgoff);
workers[i].req = (struct req *)workers[i].tblock - ntrec - 1;
}
-   mlp = [0];
-   mlp->count = 1;
-   mlp->tapea = 0;
-   mlp->firstrec = 0;
-   nextblock = mlp->tblock;
+   wp = [0];
+   wp->count = 1;
+   wp->tapea = 0;
+   wp->firstrec = 0;
+   nextblock = wp->tblock;
return(1);
 }
 
@@ -164,8 +164,8 @@ void
 writerec(char *dp, int isspcl)
 {
 
-   mlp->req[trecno].dblk = (ufs2_daddr_t)0;
-   mlp->req[trecno].count = 1;
+   wp->req[trecno].dblk = (ufs2_daddr_t)0;
+   wp->req[trecno].count = 1;
/* Can't do a structure assignment due to alignment problems */
bcopy(dp, *(nextblock)++, sizeof (union u_spcl));
if (isspcl)
@@ -185,8 +185,8 @@ dumpblock(ufs2_daddr_t blkno, int size)
dblkno = fsbtodb(sblock, blkno);
tpblks = size >> tp_bshift;
while ((avail = MIN(tpblks, ntrec - trecno)) > 0) {
-   mlp->req[trecno].dblk = dblkno;
-   mlp->req[trecno].count = avail;
+   wp->req[trecno].dblk = dblkno;
+   wp->req[trecno].count = avail;
trecno += avail;
spcl.c_tapea += avail;
if (trecno >= ntrec)
@@ -232,27 +232,27 @@ flushtape(void)
int i, blks, got;
int64_t lastfirstrec;
 
-   int siz = (char *)nextblock - (char *)mlp->req;
+   int siz = (char *)nextblock - (char *)wp->req;
 
-   mlp->req[trecno].count = 0; /* Sentinel */
+   wp->req[trecno].count = 0;  /* Sentinel */
 
-   if (atomic(write, mlp->fd, (char *)mlp->req, siz) != siz)
+   if (atomic(write, wp->fd, (char *)wp->req, siz) != siz)
quit("error writing command pipe: %s\n", strerror(errno));
-   mlp->sent = 1; /* we sent a request, read the response later */
+   wp->sent = 1; /* we sent a request, read the response later */
 
-   lastfirstrec = mlp->firstrec;
+   lastfirstrec = wp->firstrec;
 
-   if (++mlp >= [WORKERS])
-   mlp = [0];
+   if (++wp >= [WORKERS])
+   wp = [0];
 
/* Read results back from next worker */
-   if (mlp->sent) {
-   if (atomic(read, mlp->fd, (char *), sizeof got)
+   if (wp->sent) {
+   if (atomic(read, wp->fd, (char *), sizeof got)
!= sizeof got) {
perror("  DUMP: error reading command pipe in master");
dumpabort(0);
}
-   mlp->sent = 0;
+   wp->sent = 0;
 
/* Check for end of tape */
if (got < writesize) {
@@ -288,11 +288,11 @@ flushtape(void)
if (spcl.c_addr[i] != 0)
blks++;
}
-   mlp->count = lastspclrec + blks + 1 - spcl.c_tapea;
-   mlp->tapea = spcl.c_tapea;
-   mlp->firstrec = lastfirstrec + ntrec;
-   mlp->inode = curino;
-   nextblock = mlp->tblock;
+   wp->count = lastspclrec + blks + 1 - spcl.c_tapea;
+   wp->tapea = spcl.c_tapea;
+   wp->firstrec = lastfirstrec + ntrec;
+   wp->inode = curino;
+   nextblock = wp->tblock;
trecno = 0;
asize += tenths;
blockswritten += ntrec;
@@ 

Re: svn commit: r362444 - head/sbin/dump

[Colin Percival]

... never mind, it looks like Conrad got here first with r362447.

On 2020-06-20 13:12, Colin Percival wrote:
> Thanks for backing this out, Warner.
> 
> I'd like to change "slave" to "worker" here (which I think is a reasonably
> neutral and entirely inoffensive term), and in the process perhaps make some
> associated grammatical changes (since "enworker" is dubious at best).
> 
> To avoid causing any further issues: If anyone objects to the word "worker"
> please let me know in the next ~48 hours.  I think there's enough people
> reading svn-src-all that I can anticipate feedback now if anyone will care
> deeply about that word.
> 
> Colin Percival
> 
> On 2020-06-20 13:06, Warner Losh wrote:
>> Author: imp
>> Date: Sat Jun 20 20:06:14 2020
>> New Revision: 362444
>> URL: https://svnweb.freebsd.org/changeset/base/362444
>>
>> Log:
>>   Revert -r362422.
>>   
>>   While whimsical, there's too much negative energy around minion as well as 
>> the
>>   positive.
>>
>> Modified:
>>   head/sbin/dump/tape.c
>>
>> Modified: head/sbin/dump/tape.c
>> ==
>> --- head/sbin/dump/tape.cSat Jun 20 18:50:18 2020(r362443)
>> +++ head/sbin/dump/tape.cSat Jun 20 20:06:14 2020(r362444)
>> @@ -75,19 +75,19 @@ static   char *nexttape;
>>  static  FILE *popenfp = NULL;
>>  
>>  static  int atomic(ssize_t (*)(), int, char *, int);
>> -static  void dominion(int, int);
>> -static  void enminion(void);
>> +static  void doslave(int, int);
>> +static  void enslave(void);
>>  static  void flushtape(void);
>>  static  void killall(void);
>>  static  void rollforward(void);
>>  
>>  /*
>>   * Concurrent dump mods (Caltech) - disk block reading and tape writing
>> - * are exported to several minion processes.  While one minion writes the
>> + * are exported to several slave processes.  While one slave writes the
>>   * tape, the others read disk blocks; they pass control of the tape in
>>   * a ring via signals. The parent process traverses the file system and
>> - * sends writeheader()'s and lists of daddr's to the minions via pipes.
>> - * The following structure defines the instruction packets sent to minions.
>> + * sends writeheader()'s and lists of daddr's to the slaves via pipes.
>> + * The following structure defines the instruction packets sent to slaves.
>>   */
>>  struct req {
>>  ufs2_daddr_t dblk;
>> @@ -95,20 +95,20 @@ struct req {
>>  };
>>  static int reqsiz;
>>  
>> -#define MINIONS 3   /* 1 minion writing, 1 reading, 1 for slack */
>> -static struct minion {
>> +#define SLAVES 3/* 1 slave writing, 1 reading, 1 for slack */
>> +static struct slave {
>>  int64_t tapea;  /* header number at start of this chunk */
>>  int64_t firstrec;   /* record number of this block */
>>  int count;  /* count to next header (used for TS_TAPE */
>>  /* after EOT) */
>>  int inode;  /* inode that we are currently dealing with */
>> -int fd; /* FD for this minion */
>> -int pid;/* PID for this minion */
>> -int sent;   /* 1 == we've sent this minion requests */
>> +int fd; /* FD for this slave */
>> +int pid;/* PID for this slave */
>> +int sent;   /* 1 == we've sent this slave requests */
>>  char (*tblock)[TP_BSIZE]; /* buffer for data blocks */
>>  struct req *req;/* buffer for requests */
>> -} minions[MINIONS+1];
>> -static struct minion *mlp;
>> +} slaves[SLAVES+1];
>> +static struct slave *slp;
>>  
>>  static char (*nextblock)[TP_BSIZE];
>>  
>> @@ -116,9 +116,9 @@ static int master;   /* pid of master, for sending 
>> error
>>  static int tenths;  /* length of tape used per block written */
>>  static volatile sig_atomic_t caught; /* have we caught the signal to 
>> proceed? */
>>  static volatile sig_atomic_t ready; /* reached the lock point without 
>> having */
>> -/* received the SIGUSR2 signal from the prev minion? */
>> +/* received the SIGUSR2 signal from the prev slave? */
>>  static jmp_buf jmpbuf;  /* where to jump to if we are ready when the */
>> -/* SIGUSR2 arrives from the previous minion */
>> +   

Re: svn commit: r362444 - head/sbin/dump

[Colin Percival]

Thanks for backing this out, Warner.

I'd like to change "slave" to "worker" here (which I think is a reasonably
neutral and entirely inoffensive term), and in the process perhaps make some
associated grammatical changes (since "enworker" is dubious at best).

To avoid causing any further issues: If anyone objects to the word "worker"
please let me know in the next ~48 hours.  I think there's enough people
reading svn-src-all that I can anticipate feedback now if anyone will care
deeply about that word.

Colin Percival

On 2020-06-20 13:06, Warner Losh wrote:
> Author: imp
> Date: Sat Jun 20 20:06:14 2020
> New Revision: 362444
> URL: https://svnweb.freebsd.org/changeset/base/362444
> 
> Log:
>   Revert -r362422.
>   
>   While whimsical, there's too much negative energy around minion as well as 
> the
>   positive.
> 
> Modified:
>   head/sbin/dump/tape.c
> 
> Modified: head/sbin/dump/tape.c
> ==
> --- head/sbin/dump/tape.c Sat Jun 20 18:50:18 2020(r362443)
> +++ head/sbin/dump/tape.c Sat Jun 20 20:06:14 2020(r362444)
> @@ -75,19 +75,19 @@ staticchar *nexttape;
>  static   FILE *popenfp = NULL;
>  
>  static   int atomic(ssize_t (*)(), int, char *, int);
> -static   void dominion(int, int);
> -static   void enminion(void);
> +static   void doslave(int, int);
> +static   void enslave(void);
>  static   void flushtape(void);
>  static   void killall(void);
>  static   void rollforward(void);
>  
>  /*
>   * Concurrent dump mods (Caltech) - disk block reading and tape writing
> - * are exported to several minion processes.  While one minion writes the
> + * are exported to several slave processes.  While one slave writes the
>   * tape, the others read disk blocks; they pass control of the tape in
>   * a ring via signals. The parent process traverses the file system and
> - * sends writeheader()'s and lists of daddr's to the minions via pipes.
> - * The following structure defines the instruction packets sent to minions.
> + * sends writeheader()'s and lists of daddr's to the slaves via pipes.
> + * The following structure defines the instruction packets sent to slaves.
>   */
>  struct req {
>   ufs2_daddr_t dblk;
> @@ -95,20 +95,20 @@ struct req {
>  };
>  static int reqsiz;
>  
> -#define MINIONS 3/* 1 minion writing, 1 reading, 1 for slack */
> -static struct minion {
> +#define SLAVES 3 /* 1 slave writing, 1 reading, 1 for slack */
> +static struct slave {
>   int64_t tapea;  /* header number at start of this chunk */
>   int64_t firstrec;   /* record number of this block */
>   int count;  /* count to next header (used for TS_TAPE */
>   /* after EOT) */
>   int inode;  /* inode that we are currently dealing with */
> - int fd; /* FD for this minion */
> - int pid;/* PID for this minion */
> - int sent;   /* 1 == we've sent this minion requests */
> + int fd; /* FD for this slave */
> + int pid;/* PID for this slave */
> + int sent;   /* 1 == we've sent this slave requests */
>   char (*tblock)[TP_BSIZE]; /* buffer for data blocks */
>   struct req *req;/* buffer for requests */
> -} minions[MINIONS+1];
> -static struct minion *mlp;
> +} slaves[SLAVES+1];
> +static struct slave *slp;
>  
>  static char  (*nextblock)[TP_BSIZE];
>  
> @@ -116,9 +116,9 @@ static int master;/* pid of master, for sending 
> error
>  static int tenths;   /* length of tape used per block written */
>  static volatile sig_atomic_t caught; /* have we caught the signal to 
> proceed? */
>  static volatile sig_atomic_t ready; /* reached the lock point without having 
> */
> - /* received the SIGUSR2 signal from the prev minion? */
> + /* received the SIGUSR2 signal from the prev slave? */
>  static jmp_buf jmpbuf;   /* where to jump to if we are ready when the */
> - /* SIGUSR2 arrives from the previous minion */
> + /* SIGUSR2 arrives from the previous slave */
>  
>  int
>  alloctape(void)
> @@ -143,20 +143,20 @@ alloctape(void)
>* packets, so flushtape() can write them together with one write().
>* Align tape buffer on page boundary to speed up tape write().
>*/
> - for (i = 0; i <= MINIONS; i++) {
> + for (i = 0; i <= SLAVES; i++) {
>   buf = (char *)
>   malloc((unsigned)(reqsiz 

svn commit: r361150 - head/release/tools

[Colin Percival]

Author: cperciva
Date: Mon May 18 02:14:25 2020
New Revision: 361150
URL: https://svnweb.freebsd.org/changeset/base/361150

Log:
  Add ebsnvme-id to EC2 AMIs and enable /dev/aws/disk
  
  The ebsnvme-id utility exposes information about EC2 disks -- for
  Elastic Block Store volumes, their volume IDs and "linux device
  names", and for Instance Store (aka "Ephemeral") disks, their
  serial numbers.
  
  The dev_aws_disk rc.d script and associated devd.conf rule maintains
  a tree under /dev/aws/disk:
/dev/aws/disk/ebs/
/dev/aws/disk/linuxname/
/dev/aws/disk/ephemeral/
  which are symlinks to the corresponding nda or nvd devices.
  
  MFC after:1 week
  Sponsored by: https://www.patreon.com/cperciva

Modified:
  head/release/tools/ec2.conf

Modified: head/release/tools/ec2.conf
==
--- head/release/tools/ec2.conf Mon May 18 01:48:55 2020(r361149)
+++ head/release/tools/ec2.conf Mon May 18 02:14:25 2020(r361150)
@@ -6,7 +6,7 @@
 # Packages to install into the image we're creating.  This is a deliberately
 # minimalist set, providing only the packages necessary to bootstrap further
 # package installation as specified via EC2 user-data.
-export VM_EXTRA_PACKAGES="ec2-scripts firstboot-freebsd-update firstboot-pkgs 
dual-dhclient-daemon"
+export VM_EXTRA_PACKAGES="ec2-scripts firstboot-freebsd-update firstboot-pkgs 
dual-dhclient-daemon ebsnvme-id"
 
 # Include the amazon-ssm-agent package in amd64 images, since some users want
 # to be able to use it on systems which are not connected to the Internet.
@@ -17,7 +17,7 @@ if [ "${TARGET_ARCH}" = "amd64" ]; then
 fi
 
 # Set to a list of third-party software to enable in rc.conf(5).
-export VM_RC_LIST="ec2_configinit ec2_fetchkey ec2_loghostkey 
firstboot_freebsd_update firstboot_pkgs ntpd"
+export VM_RC_LIST="ec2_configinit ec2_fetchkey ec2_loghostkey 
firstboot_freebsd_update firstboot_pkgs ntpd dev_aws_disk"
 
 # Build with a 3.9 GB UFS partition; the growfs rc.d script will expand
 # the partition to fill the root disk after the EC2 instance is launched.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r361143 - head/release/tools

[Colin Percival]

On 2020-05-17 16:48, Oliver Pinter wrote:
> On Sunday, May 17, 2020, Colin Percival  <mailto:cperc...@freebsd.org>> wrote:
> +REGION=`fetch -qo-
> http://169.254.169.254/latest/meta-data/placement/availability-zone
> <http://169.254.169.254/latest/meta-data/placement/availability-zone> |
> sed -e 's/[a-z]$//'`
> 
> What will be this hard-coded ip address without any verification or at least
> https?

That's a magic IP address which connects to the EC2 "instance metadata
service".  It doesn't actually go out over the network.

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r361143 - head/release/tools

[Colin Percival]

Author: cperciva
Date: Sun May 17 21:54:59 2020
New Revision: 361143
URL: https://svnweb.freebsd.org/changeset/base/361143

Log:
  Add /etc/autofs/special_efs to EC2 AMIs
  
  Since Amazon Elastic File System is only available within AWS, it seems
  more appropriate to have this added only in EC2 AMIs rather than
  "polluting" non-EC2 images with it.
  
  Reviewed by:  gjb
  MFC after:7 days
  Relnotes: Amazon EFS filesystems can be automounted by enabling autofs
and placing "/efs -efs" into /etc/auto_master.
  Sponsored by: https://www.patreon.com/cperciva
  Differential Revision:https://reviews.freebsd.org/D24791

Modified:
  head/release/tools/ec2.conf

Modified: head/release/tools/ec2.conf
==
--- head/release/tools/ec2.conf Sun May 17 21:29:45 2020(r361142)
+++ head/release/tools/ec2.conf Sun May 17 21:54:59 2020(r361143)
@@ -113,6 +113,23 @@ vm_extra_pre_umount() {
-e '1,/^#server/s/^#server.*/server 169.254.169.123 iburst/' \
${DESTDIR}/etc/ntp.conf
 
+   # Provide a map for accessing Elastic File System mounts
+   cat > ${DESTDIR}/etc/autofs/special_efs <<'EOF'
+#!/bin/sh
+
+if [ $# -eq 0 ]; then
+# No way to know which EFS filesystems exist and are
+# accessible to this EC2 instance.
+exit 0
+fi
+
+# Provide instructions on how to mount the requested filesystem.
+FS=$1
+REGION=`fetch -qo- 
http://169.254.169.254/latest/meta-data/placement/availability-zone | sed -e 
's/[a-z]$//'`
+echo "-nfsv4,minorversion=1,oneopenown ${FS}.efs.${REGION}.amazonaws.com:/"
+EOF
+   chmod 755 ${DESTDIR}/etc/autofs/special_efs
+
# The first time the AMI boots, the installed "first boot" scripts
# should be allowed to run:
# * ec2_configinit (download and process EC2 user-data)
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r361114 - head/libexec/rc/rc.d

[Colin Percival]

On 2020-05-16 11:37, Colin Percival wrote:
> Author: cperciva
> Date: Sat May 16 18:37:48 2020
> New Revision: 361114
> URL: https://svnweb.freebsd.org/changeset/base/361114
> 
> Log:
>   Move the devmatch rc.d script before netif in the boot process.
>   
>   Prior to this change, using lagg to aggregate wired and wireless networks
>   was broken in the (relatively common) case where wifi drivers + firmware
>   are loaded by devmatch, since the interface didn't exist at the time when
>   the lagg interface was being created.
>   
>   Suggested by:   imp
>   MFC after:  3 days
PR: 246506

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r361114 - head/libexec/rc/rc.d

[Colin Percival]

Author: cperciva
Date: Sat May 16 18:37:48 2020
New Revision: 361114
URL: https://svnweb.freebsd.org/changeset/base/361114

Log:
  Move the devmatch rc.d script before netif in the boot process.
  
  Prior to this change, using lagg to aggregate wired and wireless networks
  was broken in the (relatively common) case where wifi drivers + firmware
  are loaded by devmatch, since the interface didn't exist at the time when
  the lagg interface was being created.
  
  Suggested by: imp
  MFC after:3 days

Modified:
  head/libexec/rc/rc.d/devmatch

Modified: head/libexec/rc/rc.d/devmatch
==
--- head/libexec/rc/rc.d/devmatch   Sat May 16 17:05:44 2020
(r361113)
+++ head/libexec/rc/rc.d/devmatch   Sat May 16 18:37:48 2020
(r361114)
@@ -27,6 +27,7 @@
 #
 # PROVIDE: devmatch
 # REQUIRE: kldxref
+# BEFORE: netif
 # KEYWORD: nojail
 
 . /etc/rc.subr
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r361097 - head/sys/dev/acpica

[Colin Percival]

Author: cperciva
Date: Sat May 16 01:50:28 2020
New Revision: 361097
URL: https://svnweb.freebsd.org/changeset/base/361097

Log:
  Send Lid status notification via devd from acpi_lid_status_update.
  
  Some laptops don't send ACPI "lid status changed" notifications upon
  opening the lid if the system was currently suspended.  In r358219
  this was partially fixed, updating the "lid_status" variable upon
  resume even if there is no "status changed" notification from ACPI.
  
  Unfortunately the fix in r358219 did not include notifying userland
  via devd; this causes problems on systems using upowerd (e.g. KDE),
  since upowerd remembers the most recent devd notification about the
  lid status rather than querying the sysctl to get the current status.
  
  This showed up as two symptoms when KDE's "When laptop lid closed: Sleep"
  option is set:
  1. 50% of the time, closing the lid would not trigger S3 sleep.
  2. 50% of the time, plugging/unplugging AC power would trigger S3 sleep.
  
  PR:   246477
  MFC after:3 days

Modified:
  head/sys/dev/acpica/acpi_lid.c

Modified: head/sys/dev/acpica/acpi_lid.c
==
--- head/sys/dev/acpica/acpi_lid.c  Sat May 16 00:28:49 2020
(r361096)
+++ head/sys/dev/acpica/acpi_lid.c  Sat May 16 01:50:28 2020
(r361097)
@@ -114,6 +114,9 @@ acpi_lid_status_update(struct acpi_lid_softc *sc)
/* range check value */
sc->lid_status = lid_status ? 1 : 0;
 
+   /* Send notification via devd */
+   acpi_UserNotify("Lid", sc->lid_handle, sc->lid_status);
+
 #ifdef EVDEV_SUPPORT
/* Notify evdev about lid status */
evdev_push_sw(sc->lid_evdev, SW_LID, lid_status ? 0 : 1);
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r360508 - head/sys/cam/nvme

[Colin Percival]

On 2020-04-30 14:08, Warner Losh wrote:
> Author: imp
> Date: Thu Apr 30 21:08:59 2020
> New Revision: 360508
> URL: https://svnweb.freebsd.org/changeset/base/360508
> 
>   - Unlock the periph before returning. We don't need to relock it to
> release the ccb.

You sure about that?  I'm getting a panic here:

panic: mutex CAM device lock not owned at /usr/src/sys/cam/cam_xpt.c:3983
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfe00d8243430
vpanic() at vpanic+0x182/frame 0xfe00d8243480
panic() at panic+0x43/frame 0xfe00d82434e0
__mtx_assert() at __mtx_assert+0xb0/frame 0xfe00d82434f0
xpt_release_ccb() at xpt_release_ccb+0x30/frame 0xfe00d8243520
ndaioctl() at ndaioctl+0x241/frame 0xfe00d8243730

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r360396 - head/release/tools

[Colin Percival]

Author: cperciva
Date: Mon Apr 27 21:44:02 2020
New Revision: 360396
URL: https://svnweb.freebsd.org/changeset/base/360396

Log:
  Set use_nvd=0 in EC2 AMIs.
  
  FreeBSD is in the process of switching from nvd(4) to nda(4) as the disk
  device front-end to NVMe. Changing the default in the kernel is tricky
  since existing systems may have /dev/nvd* hard-coded e.g. in /etc/fstab;
  however, there's no reason to not change the default in HEAD for *new*
  systems.
  
  At present I have no intention of MFCing this to stable branches, since
  someone might reasonably expect scripts they use for launching and
  configuring FreeBSD 12.1 instances to work with FreeBSD 12.2 AMIs, for
  example.
  
  Reviewed by:  gjb, imp
  Relnotes: NVMe disks in EC2 instances launched from 13.0 and later
now show up as nda(4) devices.
  Differential Revision:https://reviews.freebsd.org/D24583

Modified:
  head/release/tools/ec2.conf

Modified: head/release/tools/ec2.conf
==
--- head/release/tools/ec2.conf Mon Apr 27 21:41:00 2020(r360395)
+++ head/release/tools/ec2.conf Mon Apr 27 21:44:02 2020(r360396)
@@ -98,6 +98,10 @@ vm_extra_pre_umount() {
# Load the kernel module for the Amazon "Elastic Network Adapter"
echo 'if_ena_load="YES"' >> ${DESTDIR}/boot/loader.conf
 
+   # Use the "nda" driver for accessing NVMe disks rather than the
+   # historical "nvd" driver.
+   echo 'hw.nvme.use_nvd="0"' >> ${DESTDIR}/boot/loader.conf
+
# Disable ChallengeResponseAuthentication according to EC2
# requirements.
sed -i '' -e \
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r360025 - head/sys/dev/acpica

[Colin Percival]

On 2020-04-16 23:26, Alexey Dokuchaev wrote:
> On Thu, Apr 16, 2020 at 03:12:15PM -0700, Colin Percival wrote:
>> On 2020-04-16 15:05, Oliver Pinter wrote:
>>> On Thursday, April 16, 2020, Colin Percival >> <mailto:cperc...@freebsd.org>> wrote:
>>> Log:
>>>   Alert devd when acpi_video brightness changes
>>>
>>> Please add this to release notes!
>>
>> I considered that, but so far I don't have any evidence that other
>> systems exist with the same half-working ACPI.  Unless you're aware of
>> people who are seeing the same "hw.apci.video.lcd0.brightness changes
>> but thebacklight doesn't" syndrome?
> 
> So this is more like a work-around for the problem which lies elsewhere:
> something is missing in acpi_video(4) or perhaps we need a (nonexistent)
> acpi_dell(4)?  In this case, shouldn't it be disabled by default, or at
> least be toggable since as you say, evidently just a minority of systems
> exist with the same half-working ACPI?

Having occasional extra messages coming through devctl(4) is harmless; by
default devd will just drop them on the floor since there isn't anything
configured to do with them.

In an upcoming change to the graphics/intel-backlight port (if/when someone
merges my patch into their github repo...) I'll be providing a configuration
file which can be copied into /usr/local/etc/devd/ in order to hook these
messages up to the intel-backlight utility.

So... yes, for practical purposes this is disabled by default.

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r360025 - head/sys/dev/acpica

[Colin Percival]

On 2020-04-16 15:05, Oliver Pinter wrote:
> On Thursday, April 16, 2020, Colin Percival  <mailto:cperc...@freebsd.org>> wrote:
> Log:
>   Alert devd when acpi_video brightness changes
> 
> Please add this to release notes! 
I considered that, but so far I don't have any evidence that other systems
exist with the same half-working ACPI.  Unless you're aware of people who
are seeing the same "hw.apci.video.lcd0.brightness changes but the backlight
doesn't" syndrome?

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r360025 - head/sys/dev/acpica

[Colin Percival]

Author: cperciva
Date: Thu Apr 16 21:56:52 2020
New Revision: 360025
URL: https://svnweb.freebsd.org/changeset/base/360025

Log:
  Alert devd when acpi_video brightness changes
  
  On my Dell Latitude 7390 laptop, the brightness hotkeys
  (Fn+) send ACPI notifications which acpi_video
  handles by adjusting its brightness setting; but ACPI does not
  actually do anything with the backlight.
  
  Announcing brightness changes via devd makes it possible to close
  the loop by triggering the intel_backlight utility to perform the
  required backlight adjustment.
  
  Reviewed by:  imp
  MFC after:3 days
  Differential Revision:https://reviews.freebsd.org/D24424

Modified:
  head/sys/dev/acpica/acpi_video.c

Modified: head/sys/dev/acpica/acpi_video.c
==
--- head/sys/dev/acpica/acpi_video.cThu Apr 16 21:53:17 2020
(r360024)
+++ head/sys/dev/acpica/acpi_video.cThu Apr 16 21:56:52 2020
(r360025)
@@ -1036,6 +1036,7 @@ vo_get_brightness(ACPI_HANDLE handle)
 static void
 vo_set_brightness(ACPI_HANDLE handle, int level)
 {
+   char notify_buf[16];
ACPI_STATUS status;
 
ACPI_SERIAL_ASSERT(video_output);
@@ -1043,6 +1044,8 @@ vo_set_brightness(ACPI_HANDLE handle, int level)
if (ACPI_FAILURE(status))
printf("can't evaluate %s._BCM - %s\n",
   acpi_name(handle), AcpiFormatException(status));
+   snprintf(notify_buf, sizeof(notify_buf), "notify=%d", level);
+   devctl_notify("ACPI", "Video", "brightness", notify_buf);
 }
 
 static UINT32
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r355570 - head/sys/dev/sound/pci/hda

[Colin Percival]

On 2019-12-09 11:25, Mark Johnston wrote:
> Author: markj
> Date: Mon Dec  9 19:25:15 2019
> New Revision: 355570
> URL: https://svnweb.freebsd.org/changeset/base/355570
> 
> Log:
>   Configure headphone redirection for the Dell L780 and X1 Carbon 7th gen.
>   
>   As we do for many other laptops, put the headphone jack and speakers in
>   the same association by default so that the generic sound device
>   automatically switches between them.
>   
> [...]
> - } else if (id == HDA_CODEC_ALC256 && subid == DELL_I7577_SUBVENDOR) {
> + } else if (id == HDA_CODEC_ALC256 && (subid == DELL_I7577_SUBVENDOR ||
> + subid == DELL_L7480_SUBVENDOR)) {
>   switch (nid) {
>   case 20:
>   patch = "as=1 seq=0";> [...]

This is what we need for the Dell Latitude 7390 too -- nid20 is the internal
speakers while nid33 is the headphone jack.  Can you make the change or walk
me through what needs to be done?  (I'm not sure how the _SUBVENDOR bit
works.)

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r358478 - head/usr.sbin/powerd

[Colin Percival]

Author: cperciva
Date: Sat Feb 29 22:31:23 2020
New Revision: 358478
URL: https://svnweb.freebsd.org/changeset/base/358478

Log:
  Add -N option to powerd(8) to ignore "nice" time.
  
  With powerd_flags="-N", this makes powerd(8) exclude "nice" time when
  computing the CPU utilization.  This makes it possible to prevent
  CPU-intensive "background" processes from spinning up the CPU.
  
  Note that only *userland* CPU usage belonging to "nice" processes is
  excluded; we do not track whether time spent in the kernel is on behalf
  of nice or non-nice processes, so kernel-intensive nice processes can
  still result in the CPU being sped up.
  
  MFC after:2 weeks
  Differential Revision:https://reviews.freebsd.org/D23868

Modified:
  head/usr.sbin/powerd/powerd.8
  head/usr.sbin/powerd/powerd.c

Modified: head/usr.sbin/powerd/powerd.8
==
--- head/usr.sbin/powerd/powerd.8   Sat Feb 29 22:16:27 2020
(r358477)
+++ head/usr.sbin/powerd/powerd.8   Sat Feb 29 22:31:23 2020
(r358478)
@@ -37,6 +37,7 @@
 .Op Fl i Ar percent
 .Op Fl m Ar freq
 .Op Fl M Ar freq
+.Op Fl N
 .Op Fl n Ar mode
 .Op Fl p Ar ival
 .Op Fl P Ar pidfile
@@ -102,6 +103,10 @@ The default is 50% or lower.
 Specifies the minimum frequency to throttle down to.
 .It Fl M Ar freq
 Specifies the maximum frequency to throttle up to.
+.It Fl N
+Treat "nice" time as idle for the purpose of load calculation;
+i.e. do not increase the CPU frequency if the CPU is only busy
+with "nice" processes.
 .It Fl n Ar mode
 Selects the
 .Ar mode

Modified: head/usr.sbin/powerd/powerd.c
==
--- head/usr.sbin/powerd/powerd.c   Sat Feb 29 22:16:27 2020
(r358477)
+++ head/usr.sbin/powerd/powerd.c   Sat Feb 29 22:31:23 2020
(r358478)
@@ -86,7 +86,7 @@ static const char *modes[] = {
 #define DEVDPIPE   "/var/run/devd.pipe"
 #define DEVCTL_MAXBUF  1024
 
-static int read_usage_times(int *load);
+static int read_usage_times(int *load, int nonice);
 static int read_freqs(int *numfreqs, int **freqs, int **power,
int minfreq, int maxfreq);
 static int set_freq(int freq);
@@ -135,15 +135,17 @@ static struct timeval tried_devd;
  * This function returns summary load of all CPUs.  It was made so
  * intentionally to not reduce performance in scenarios when several
  * threads are processing requests as a pipeline -- running one at
- * a time on different CPUs and waiting for each other.
+ * a time on different CPUs and waiting for each other.  If nonice
+ * is nonzero, only user+sys+intr time will be counted as load; any
+ * nice time will be treated as if idle.
  */
 static int
-read_usage_times(int *load)
+read_usage_times(int *load, int nonice)
 {
static long *cp_times = NULL, *cp_times_old = NULL;
static int ncpus = 0;
size_t cp_times_len;
-   int error, cpu, i, total;
+   int error, cpu, i, total, excl;
 
if (cp_times == NULL) {
cp_times_len = 0;
@@ -175,8 +177,12 @@ read_usage_times(int *load)
}
if (total == 0)
continue;
-   *load += 100 - (cp_times[cpu * CPUSTATES + CP_IDLE] -
-   cp_times_old[cpu * CPUSTATES + CP_IDLE]) * 100 / 
total;
+   excl = cp_times[cpu * CPUSTATES + CP_IDLE] -
+   cp_times_old[cpu * CPUSTATES + CP_IDLE];
+   if (nonice)
+   excl += cp_times[cpu * CPUSTATES + CP_NICE] -
+   cp_times_old[cpu * CPUSTATES + CP_NICE];
+   *load += 100 - excl * 100 / total;
}
}
 
@@ -473,7 +479,7 @@ usage(void)
 {
 
fprintf(stderr,
-"usage: powerd [-v] [-a mode] [-b mode] [-i %%] [-m freq] [-M freq] [-n mode] 
[-p ival] [-r %%] [-s source] [-P pidfile]\n");
+"usage: powerd [-v] [-a mode] [-b mode] [-i %%] [-m freq] [-M freq] [-N] [-n 
mode] [-p ival] [-r %%] [-s source] [-P pidfile]\n");
exit(1);
 }
 
@@ -490,6 +496,7 @@ main(int argc, char * argv[])
int ch, mode, mode_ac, mode_battery, mode_none, idle, to;
uint64_t mjoules_used;
size_t len;
+   int nonice;
 
/* Default mode for all AC states is adaptive. */
mode_ac = mode_none = MODE_HIADAPTIVE;
@@ -499,12 +506,13 @@ main(int argc, char * argv[])
poll_ival = DEFAULT_POLL_INTERVAL;
mjoules_used = 0;
vflag = 0;
+   nonice = 0;
 
/* User must be root to control frequencies. */
if (geteuid() != 0)
errx(1, "must be root to run");
 
-   while ((ch = getopt(argc, argv, "a:b:i:m:M:n:p:P:r:s:v")) != -1)
+   while ((ch = getopt(argc, argv, "a:b:i:m:M:Nn:p:P:r:s:v")) != -1)
switch (ch) {
 

svn commit: r357750 - head/release/tools

[Colin Percival]

Author: cperciva
Date: Tue Feb 11 04:05:45 2020
New Revision: 357750
URL: https://svnweb.freebsd.org/changeset/base/357750

Log:
  Mount /dev while deinstalling pkg inside EC2 AMIs.
  
  This gets rid of (harmless) warnings:
  "pkg: Cannot open /dev/null:No such file or directory".
  
  MFC after:2 weeks

Modified:
  head/release/tools/ec2.conf

Modified: head/release/tools/ec2.conf
==
--- head/release/tools/ec2.conf Tue Feb 11 04:03:22 2020(r357749)
+++ head/release/tools/ec2.conf Tue Feb 11 04:05:45 2020(r357750)
@@ -40,8 +40,10 @@ vm_extra_pre_umount() {
# catalogue and install or update pkg when the instance first
# launches, so these files would just be replaced anyway; removing
# them from the image allows it to boot faster.
+   mount -t devfs devfs ${DESTDIR}/dev
chroot ${DESTDIR} ${EMULATOR} env ASSUME_ALWAYS_YES=yes \
/usr/sbin/pkg delete -f -y pkg
+   umount ${DESTDIR}/dev
rm ${DESTDIR}/var/db/pkg/repo-*.sqlite
 
# The size of the EC2 root disk can be configured at instance launch
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r357749 - head/release/tools

[Colin Percival]

Author: cperciva
Date: Tue Feb 11 04:03:22 2020
New Revision: 357749
URL: https://svnweb.freebsd.org/changeset/base/357749

Log:
  Remove /qemu from EC2 ARM AMIs
  
  I forgot to do this as part of r345858 -- I added it to the
  vm_extra_pre_umount in vmimage.subr but forgot that function
  was overridden in the EC2 build.
  
  MFC after:2 weeks

Modified:
  head/release/tools/ec2.conf

Modified: head/release/tools/ec2.conf
==
--- head/release/tools/ec2.conf Tue Feb 11 03:37:42 2020(r357748)
+++ head/release/tools/ec2.conf Tue Feb 11 04:03:22 2020(r357749)
@@ -116,6 +116,9 @@ vm_extra_pre_umount() {
# * firstboot_pkgs (install packages)
touch ${DESTDIR}/firstboot
 
+   if ! [ -z "${QEMUSTATIC}" ]; then
+   rm -f ${DESTDIR}/${EMULATOR}
+   fi
rm -f ${DESTDIR}/etc/resolv.conf
 
return 0
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r356758 - in head/usr.sbin/bsdinstall: . scripts

[Colin Percival]

On 2020-01-15 21:07, Philip Paeps wrote:
> On 2020-01-16 04:57:28 (+1000), Oliver Pinter wrote:
>> On Wednesday, January 15, 2020, Ben Woods  wrote:
>>>   bsdinstall: Change "default" (first) Partitioning method to ZFS
> 
>> Plus I miss from here the relontes tag.
> 
> I'm not sure if this merits a release notes entry but ... sure.
> 
> There is not actually a functional change here.  It's just a defaults change.

I'd say that a change in defaults is far more deserving of being mentioned
in the release notes than, say, adding a new feature.  Nobody will trip over
new features by mistake, but there's probably someone out there who is used
to holding down the Enter key in the installer and expects to get UFS.

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r356207 - in head/sys/arm64: arm64 include

[Colin Percival]

On 2019-12-30 12:30, Alan Cox wrote:
> Author: alc
> Date: Mon Dec 30 20:30:31 2019
> New Revision: 356207
> URL: https://svnweb.freebsd.org/changeset/base/356207
> 
> Log:
>   Determine whether the MMU hardware is capable of updating a page table
>   entry's access flag and dirty state, and enable this feature when it's
>   available.

Am I right in thinking that this should result in improved performance under
load?  (I'm wondering if it's time to redo my amd64 vs arm64 EC2 benchmarks.)

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r354482 - head/sys/x86/x86

[Colin Percival]

On 2019-11-07 13:34, Andriy Gapon wrote:
> On 07/11/2019 23:19, Colin Percival wrote:
>> On 2019-11-07 13:14, Andriy Gapon wrote:
>>>   x86 stack_save_td_running() can work safely only if IPI_TRACE is a
>>>   non-maskable interrupt.  But at the moment FreeBSD/Xen does not provide
>>>   support for the NMI delivery mode.  So, mark the functionality as
>>>   unsupported similarly to other platforms without NMI.
>>> [...]
>>> +#ifdef XENHVM
>>
>> I'm not sure this does what you intended.  XENHVM simply adds *support* for
>> running under Xen/HVM, and it's part of the GENERIC kernel.
> 
> Oh... I should not have rushed.
> So, I should really check for a run-time Xen marker?

I'm guessing that you want

#include 

...

if (vm_guest == VM_GUEST_XEN)
return (EOPNOTSUPP);

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r354482 - head/sys/x86/x86

[Colin Percival]

On 2019-11-07 13:14, Andriy Gapon wrote:
>   x86 stack_save_td_running() can work safely only if IPI_TRACE is a
>   non-maskable interrupt.  But at the moment FreeBSD/Xen does not provide
>   support for the NMI delivery mode.  So, mark the functionality as
>   unsupported similarly to other platforms without NMI.
> [...]
> +#ifdef XENHVM

I'm not sure this does what you intended.  XENHVM simply adds *support* for
running under Xen/HVM, and it's part of the GENERIC kernel.

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r353022 - head/release/tools

[Colin Percival]

Author: cperciva
Date: Wed Oct  2 21:35:39 2019
New Revision: 353022
URL: https://svnweb.freebsd.org/changeset/base/353022

Log:
  Switch EC2 AMIs from using the dual-dhclient script to using the new
  dual-dhclient-daemon daemon.  This makes it possible to stop/restart
  the dhclients.
  
  MFC after:1 month

Modified:
  head/release/tools/ec2.conf

Modified: head/release/tools/ec2.conf
==
--- head/release/tools/ec2.conf Wed Oct  2 21:01:23 2019(r353021)
+++ head/release/tools/ec2.conf Wed Oct  2 21:35:39 2019(r353022)
@@ -6,7 +6,7 @@
 # Packages to install into the image we're creating.  This is a deliberately
 # minimalist set, providing only the packages necessary to bootstrap further
 # package installation as specified via EC2 user-data.
-export VM_EXTRA_PACKAGES="ec2-scripts firstboot-freebsd-update firstboot-pkgs 
dual-dhclient"
+export VM_EXTRA_PACKAGES="ec2-scripts firstboot-freebsd-update firstboot-pkgs 
dual-dhclient-daemon"
 
 # Include the amazon-ssm-agent package in amd64 images, since some users want
 # to be able to use it on systems which are not connected to the Internet.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r346628 - head/usr.sbin/pkg

[Colin Percival]

Author: cperciva
Date: Wed Apr 24 06:25:21 2019
New Revision: 346628
URL: https://svnweb.freebsd.org/changeset/base/346628

Log:
  Keep two versions of the FreeBSD.conf pkg configuration file; one which
  points at the "latest" branch and one which points at the "quarterly"
  branch.  Install the "latest" version unless overridden via the newly
  added PKGCONFBRANCH variable.
  
  This does not change user-visible behaviour (assuming said vairable is
  not set) but will make it easier to change the defaults in the future --
  on stable branches we will want "latest" on x86 but "quarterly" elsewhere.
  
  Discussed with:   gjb
  MFC after:3 days
  X-MFC:After MFCing this I'll make a direct commit to stable/* 
to
switch non-x86 architectures to "quarterly".

Added:
  head/usr.sbin/pkg/FreeBSD.conf.latest
 - copied unchanged from r346627, head/usr.sbin/pkg/FreeBSD.conf
  head/usr.sbin/pkg/FreeBSD.conf.quarterly
 - copied, changed from r346627, head/usr.sbin/pkg/FreeBSD.conf
Deleted:
  head/usr.sbin/pkg/FreeBSD.conf
Modified:
  head/usr.sbin/pkg/Makefile

Copied: head/usr.sbin/pkg/FreeBSD.conf.latest (from r346627, 
head/usr.sbin/pkg/FreeBSD.conf)
==
--- /dev/null   00:00:00 1970   (empty, because file is newly added)
+++ head/usr.sbin/pkg/FreeBSD.conf.latest   Wed Apr 24 06:25:21 2019
(r346628, copy of r346627, head/usr.sbin/pkg/FreeBSD.conf)
@@ -0,0 +1,16 @@
+# $FreeBSD$
+#
+# To disable this repository, instead of modifying or removing this file,
+# create a /usr/local/etc/pkg/repos/FreeBSD.conf file:
+#
+#   mkdir -p /usr/local/etc/pkg/repos
+#   echo "FreeBSD: { enabled: no }" > /usr/local/etc/pkg/repos/FreeBSD.conf
+#
+
+FreeBSD: {
+  url: "pkg+http://pkg.FreeBSD.org/${ABI}/latest;,
+  mirror_type: "srv",
+  signature_type: "fingerprints",
+  fingerprints: "/usr/share/keys/pkg",
+  enabled: yes
+}

Copied and modified: head/usr.sbin/pkg/FreeBSD.conf.quarterly (from r346627, 
head/usr.sbin/pkg/FreeBSD.conf)
==
--- head/usr.sbin/pkg/FreeBSD.conf  Wed Apr 24 05:52:24 2019
(r346627, copy source)
+++ head/usr.sbin/pkg/FreeBSD.conf.quarterlyWed Apr 24 06:25:21 2019
(r346628)
@@ -8,7 +8,7 @@
 #
 
 FreeBSD: {
-  url: "pkg+http://pkg.FreeBSD.org/${ABI}/latest;,
+  url: "pkg+http://pkg.FreeBSD.org/${ABI}/quarterly;,
   mirror_type: "srv",
   signature_type: "fingerprints",
   fingerprints: "/usr/share/keys/pkg",

Modified: head/usr.sbin/pkg/Makefile
==
--- head/usr.sbin/pkg/Makefile  Wed Apr 24 05:52:24 2019(r346627)
+++ head/usr.sbin/pkg/Makefile  Wed Apr 24 06:25:21 2019(r346628)
@@ -1,6 +1,8 @@
 # $FreeBSD$
 
-CONFS= FreeBSD.conf
+PKGCONFBRANCH?=latest
+CONFS= FreeBSD.conf.${PKGCONFBRANCH}
+CONFSNAME= FreeBSD.conf
 CONFSDIR=  /etc/pkg
 CONFSMODE= 644
 PROG=  pkg


___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r345858 - in head/release: . tools

[Colin Percival]

Author: cperciva
Date: Wed Apr  3 21:54:47 2019
New Revision: 345858
URL: https://svnweb.freebsd.org/changeset/base/345858

Log:
  Add support for cross-building cloudware images.
  
  If MACHINE_ARCH doesn't match TARGET_ARCH, and we're not in the special
  case of building i386 images on an amd64 host, we need to pull in the
  qemu-user-static package; this allows us to run some commands inside
  the VM disk image chroot, most notably to install packages.
  
  Reviewed by:  gjb
  MFC after:2 weeks
  Sponsored by: FreeBSD/EC2 patreon (https://www.patreon.com/cperciva)

Modified:
  head/release/Makefile.vm
  head/release/tools/ec2.conf
  head/release/tools/vmimage.subr

Modified: head/release/Makefile.vm
==
--- head/release/Makefile.vmWed Apr  3 21:48:09 2019(r345857)
+++ head/release/Makefile.vmWed Apr  3 21:54:47 2019(r345858)
@@ -39,6 +39,24 @@ VAGRANT-VMWARE_FORMAT=   vmdk
 VAGRANT-VMWARE_DESC=   Vagrant Image for VMWare
 VAGRANT-VMWARE_DISK=   ${OSRELEASE}.vmware.${VAGRANT_FORMAT}
 
+emulator-portinstall:
+.if ${TARGET_ARCH} != ${MACHINE_ARCH}
+.if ( ${TARGET_ARCH} != "i386" ) || ( ${MACHINE_ARCH} != "amd64" )
+.if !exists(/usr/local/bin/qemu-${TARGET_ARCH}-static)
+.if exists(${PORTSDIR}/emulators/qemu-user-static/Makefile)
+   env - PATH=$$PATH make -C ${PORTSDIR}/emulators/qemu-user-static 
BATCH=1 all install clean
+.else
+.if !exists(/usr/local/sbin/pkg-static)
+   env ASSUME_ALWAYS_YES=yes pkg bootstrap -y
+.endif
+   env ASSUME_ALWAYS_YES=yes pkg install -y emulators/qemu-user-static
+.endif
+.endif
+
+QEMUSTATIC=/usr/local/bin/qemu-${TARGET_ARCH}-static
+.endif
+.endif
+
 .if defined(WITH_CLOUDWARE) && !empty(WITH_CLOUDWARE) && !empty(CLOUDWARE)
 . for _CW in ${CLOUDWARE}
 CLOUDTARGETS+= cw-${_CW:tl}
@@ -53,9 +71,10 @@ ${_CW:tu}IMAGE=  ${_CW:tl}.${${_CW:tu}_FORMAT}
 ${_CW:tu}CONF?=${.CURDIR}/tools/${_CW:tl}.conf
 .  endif
 
-cw-${_CW:tl}:
+cw-${_CW:tl}: emulator-portinstall
mkdir -p ${.OBJDIR}/${.TARGET}
env TARGET=${TARGET} TARGET_ARCH=${TARGET_ARCH} SWAPSIZE=${SWAPSIZE} \
+   QEMUSTATIC=${QEMUSTATIC} \
${.CURDIR}/scripts/mk-vmimage.sh \
-C ${.CURDIR}/tools/vmimage.subr -d ${.OBJDIR}/${.TARGET} \
-i ${.OBJDIR}/${_CW:tl}.img -s ${VMSIZE} -f ${${_CW:tu}_FORMAT} 
\

Modified: head/release/tools/ec2.conf
==
--- head/release/tools/ec2.conf Wed Apr  3 21:48:09 2019(r345857)
+++ head/release/tools/ec2.conf Wed Apr  3 21:54:47 2019(r345858)
@@ -40,7 +40,7 @@ vm_extra_pre_umount() {
# catalogue and install or update pkg when the instance first
# launches, so these files would just be replaced anyway; removing
# them from the image allows it to boot faster.
-   chroot ${DESTDIR} env ASSUME_ALWAYS_YES=yes \
+   chroot ${DESTDIR} ${EMULATOR} env ASSUME_ALWAYS_YES=yes \
/usr/sbin/pkg delete -f -y pkg
rm ${DESTDIR}/var/db/pkg/repo-*.sqlite
 

Modified: head/release/tools/vmimage.subr
==
--- head/release/tools/vmimage.subr Wed Apr  3 21:48:09 2019
(r345857)
+++ head/release/tools/vmimage.subr Wed Apr  3 21:54:47 2019
(r345858)
@@ -148,10 +148,15 @@ vm_install_base() {
hostname="$(echo $(uname -o) | tr '[:upper:]' '[:lower:]')"
echo "hostname=\"${hostname}\"" >> ${DESTDIR}/etc/rc.conf
 
+   if ! [ -z "${QEMUSTATIC}" ]; then
+   export EMULATOR=/qemu
+   cp ${QEMUSTATIC} ${DESTDIR}/${EMULATOR}
+   fi
+
mkdir -p ${DESTDIR}/dev
mount -t devfs devfs ${DESTDIR}/dev
-   chroot ${DESTDIR} /usr/bin/newaliases
-   chroot ${DESTDIR} /etc/rc.d/ldconfig forcestart
+   chroot ${DESTDIR} ${EMULATOR} /usr/bin/newaliases
+   chroot ${DESTDIR} ${EMULATOR} /bin/sh /etc/rc.d/ldconfig forcestart
umount_loop ${DESTDIR}/dev
 
cp /etc/resolv.conf ${DESTDIR}/etc/resolv.conf
@@ -188,9 +193,9 @@ vm_extra_install_packages() {
fi
mkdir -p ${DESTDIR}/dev
mount -t devfs devfs ${DESTDIR}/dev
-   chroot ${DESTDIR} env ASSUME_ALWAYS_YES=yes \
+   chroot ${DESTDIR} ${EMULATOR} env ASSUME_ALWAYS_YES=yes \
/usr/sbin/pkg bootstrap -y
-   chroot ${DESTDIR} env ASSUME_ALWAYS_YES=yes \
+   chroot ${DESTDIR} ${EMULATOR} env ASSUME_ALWAYS_YES=yes \
/usr/sbin/pkg install -y ${VM_EXTRA_PACKAGES}
umount_loop ${DESTDIR}/dev
 
@@ -210,13 +215,16 @@ vm_extra_pre_umount() {
# Note: When overriding this function, removing resolv.conf in the
# disk image must be included.
 
+   if ! [ -z "${QEMUSTATIC}" ]; then
+   rm -f ${DESTDIR}/${EMULATOR}
+   fi
rm -f ${DESTDIR}/etc/resolv.conf

svn commit: r348681 - head/sys/dev/pci

[Colin Percival]

Author: cperciva
Date: Wed Jun  5 04:58:42 2019
New Revision: 348681
URL: https://svnweb.freebsd.org/changeset/base/348681

Log:
  Only respond to the PCIe Attention Button if a device is already plugged in.
  
  Prior to this commit, if PCIEM_SLOT_STA_ABP and PCIEM_SLOT_STA_PDC are
  asserted simultaneously, FreeBSD sets a 5 second "hardware going away" timer
  and then processes the "presence detect" change. In the (physically
  challenging) case that someone presses the "attention button" and inserts
  a new PCIe device at exactly the same moment, this results in FreeBSD
  recognizing that the device is present, attaching it, and then detaching it
  5 seconds later.
  
  On EC2 "bare metal" hardware this is the precise sequence of events which
  takes place when a new EBS volume is attached; virtual machines have no
  difficulty effecting physically implausible simultaneity.
  
  This patch changes the handling of PCIEM_SLOT_STA_ABP to only detach a
  device if the presence of a device was detected *before* the interrupt
  which reports the Attention Button push.
  
  Reported by:  Matt Wilson
  Reviewed by:  jhb
  MFC after:1 week
  Sponsored by: https://www.patreon.com/cperciva
  Differential Revision:https://reviews.freebsd.org/D20499

Modified:
  head/sys/dev/pci/pci_pci.c

Modified: head/sys/dev/pci/pci_pci.c
==
--- head/sys/dev/pci/pci_pci.c  Wed Jun  5 04:01:31 2019(r348680)
+++ head/sys/dev/pci/pci_pci.c  Wed Jun  5 04:58:42 2019(r348681)
@@ -1170,9 +1170,11 @@ pcib_pcie_intr_hotplug(void *arg)
 {
struct pcib_softc *sc;
device_t dev;
+   uint16_t old_slot_sta;
 
sc = arg;
dev = sc->dev;
+   old_slot_sta = sc->pcie_slot_sta;
sc->pcie_slot_sta = pcie_read_config(dev, PCIER_SLOT_STA, 2);
 
/* Clear the events just reported. */
@@ -1188,7 +1190,8 @@ pcib_pcie_intr_hotplug(void *arg)
"Attention Button Pressed: Detach Cancelled\n");
sc->flags &= ~PCIB_DETACH_PENDING;
callout_stop(>pcie_ab_timer);
-   } else {
+   } else if (old_slot_sta & PCIEM_SLOT_STA_PDS) {
+   /* Only initiate detach sequence if device present. */
device_printf(dev,
"Attention Button Pressed: Detaching in 5 seconds\n");
sc->flags |= PCIB_DETACH_PENDING;
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r348195 - in head/sys: conf dev/uart

[Colin Percival]

Author: cperciva
Date: Thu May 23 19:55:53 2019
New Revision: 348195
URL: https://svnweb.freebsd.org/changeset/base/348195

Log:
  Use ACPI SPCR on x86
  
  This takes the SPCR code currently in uart_cpu_arm64.c, moves it into
  a new uart_cpu_acpi.c (with some associated refactoring), and uses it
  from both arm64 and x86.
  
  An SPCR serial port address AccessWidth field value of 0 ("reserved")
  is now treated as 1 ("byte access") in order to work around a buggy
  SPCR table on Amazon EC2 i3.metal instances.
  
  Reviewed by:  manu, Greg V
  MFC after:3 days
  Sponsored by: https://www.patreon.com/cperciva
  Differential Revision:https://reviews.freebsd.org/D20357

Added:
  head/sys/dev/uart/uart_cpu_acpi.c   (contents, props changed)
Modified:
  head/sys/conf/files
  head/sys/dev/uart/uart_cpu_acpi.h
  head/sys/dev/uart/uart_cpu_arm64.c
  head/sys/dev/uart/uart_cpu_x86.c

Modified: head/sys/conf/files
==
--- head/sys/conf/files Thu May 23 19:26:50 2019(r348194)
+++ head/sys/conf/files Thu May 23 19:55:53 2019(r348195)
@@ -3164,6 +3164,7 @@ dev/uart/uart_bus_pci.c   optional uart pci
 dev/uart/uart_bus_puc.coptional uart puc
 dev/uart/uart_bus_scc.coptional uart scc
 dev/uart/uart_core.c   optional uart
+dev/uart/uart_cpu_acpi.c   optional uart acpi
 dev/uart/uart_dbg.coptional uart gdb
 dev/uart/uart_dev_msm.coptional uart uart_msm fdt
 dev/uart/uart_dev_mvebu.c  optional uart uart_mvebu

Added: head/sys/dev/uart/uart_cpu_acpi.c
==
--- /dev/null   00:00:00 1970   (empty, because file is newly added)
+++ head/sys/dev/uart/uart_cpu_acpi.c   Thu May 23 19:55:53 2019
(r348195)
@@ -0,0 +1,167 @@
+/*-
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
+ *
+ * Copyright (c) 2016 The FreeBSD Foundation
+ * Copyright (c) 2019 Colin Percival
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * 1. Redistributions of source code must retain the above copyright
+ *notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in the
+ *documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
+ * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+ * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
+ * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
+ * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
+ * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include 
+__FBSDID("$FreeBSD$");
+
+#include 
+#include 
+#include 
+
+#include 
+
+#include 
+#include 
+#include 
+#include 
+
+#include 
+#include 
+#include 
+
+extern bus_space_tag_t uart_bus_space_io;
+extern bus_space_tag_t uart_bus_space_mem;
+
+static struct acpi_uart_compat_data *
+uart_cpu_acpi_scan(uint8_t interface_type)
+{
+   struct acpi_uart_compat_data **cd, *curcd;
+   int i;
+
+   SET_FOREACH(cd, uart_acpi_class_and_device_set) {
+   curcd = *cd;
+   for (i = 0; curcd[i].cd_hid != NULL; i++) {
+   if (curcd[i].cd_port_subtype == interface_type)
+   return ([i]);
+   }
+   }
+
+   SET_FOREACH(cd, uart_acpi_class_set) {
+   curcd = *cd;
+   for (i = 0; curcd[i].cd_hid != NULL; i++) {
+   if (curcd[i].cd_port_subtype == interface_type)
+   return ([i]);
+   }
+   }
+
+   return (NULL);
+}
+
+int
+uart_cpu_acpi_spcr(int devtype, struct uart_devinfo *di)
+{
+   vm_paddr_t spcr_physaddr;
+   ACPI_TABLE_SPCR *spcr;
+   struct acpi_uart_compat_data *cd;
+   struct uart_class *class;
+   int error = ENXIO;
+
+   /* SPCR only tells us about consoles. */
+   if (devtype != UART_DEV_CONSOLE)
+   return (error);
+
+   /* Look for the SPCR table. */
+   spcr_physaddr = acpi_find_table(ACPI_SIG_SPCR);
+   if (spcr_physaddr == 0)
+   return (error);
+   spcr = acpi_map_table(spcr_physaddr, ACPI_SIG_SPCR);
+   if (spcr == NULL) {

svn commit: r347370 - head/release/tools

[Colin Percival]

Author: cperciva
Date: Wed May  8 21:03:03 2019
New Revision: 347370
URL: https://svnweb.freebsd.org/changeset/base/347370

Log:
  Bump EC2 AMI filesystem size up to 4000 MB.
  
  AMIs have been breaking for the past month due to insufficient disk space.
  
  Due to the small amount of overhead in the disk image, the EC2 AMIs end
  up with the same (4GB) minimum disk size.
  
  Reported by:  Michal Krawczyk

Modified:
  head/release/tools/ec2.conf

Modified: head/release/tools/ec2.conf
==
--- head/release/tools/ec2.conf Wed May  8 20:18:11 2019(r347369)
+++ head/release/tools/ec2.conf Wed May  8 21:03:03 2019(r347370)
@@ -19,12 +19,12 @@ fi
 # Set to a list of third-party software to enable in rc.conf(5).
 export VM_RC_LIST="ec2_configinit ec2_fetchkey ec2_loghostkey 
firstboot_freebsd_update firstboot_pkgs ntpd"
 
-# Build with a 3 GB UFS partition; the growfs rc.d script will expand
+# Build with a 3.9 GB UFS partition; the growfs rc.d script will expand
 # the partition to fill the root disk after the EC2 instance is launched.
 # Note that if this is set to G, we will end up with an  GB disk
 # image since VMSIZE is the size of the UFS partition, not the disk which
 # it resides within.
-export VMSIZE=3072M
+export VMSIZE=4000M
 
 # No swap space; the ec2_ephemeralswap rc.d script will allocate swap
 # space on EC2 ephemeral disks.  (If they exist -- the T2 low-cost instances
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r346628 - head/usr.sbin/pkg

[Colin Percival]

Author: cperciva
Date: Wed Apr 24 06:25:21 2019
New Revision: 346628
URL: https://svnweb.freebsd.org/changeset/base/346628

Log:
  Keep two versions of the FreeBSD.conf pkg configuration file; one which
  points at the "latest" branch and one which points at the "quarterly"
  branch.  Install the "latest" version unless overridden via the newly
  added PKGCONFBRANCH variable.
  
  This does not change user-visible behaviour (assuming said vairable is
  not set) but will make it easier to change the defaults in the future --
  on stable branches we will want "latest" on x86 but "quarterly" elsewhere.
  
  Discussed with:   gjb
  MFC after:3 days
  X-MFC:After MFCing this I'll make a direct commit to stable/* 
to
switch non-x86 architectures to "quarterly".

Added:
  head/usr.sbin/pkg/FreeBSD.conf.latest
 - copied unchanged from r346627, head/usr.sbin/pkg/FreeBSD.conf
  head/usr.sbin/pkg/FreeBSD.conf.quarterly
 - copied, changed from r346627, head/usr.sbin/pkg/FreeBSD.conf
Deleted:
  head/usr.sbin/pkg/FreeBSD.conf
Modified:
  head/usr.sbin/pkg/Makefile

Copied: head/usr.sbin/pkg/FreeBSD.conf.latest (from r346627, 
head/usr.sbin/pkg/FreeBSD.conf)
==
--- /dev/null   00:00:00 1970   (empty, because file is newly added)
+++ head/usr.sbin/pkg/FreeBSD.conf.latest   Wed Apr 24 06:25:21 2019
(r346628, copy of r346627, head/usr.sbin/pkg/FreeBSD.conf)
@@ -0,0 +1,16 @@
+# $FreeBSD$
+#
+# To disable this repository, instead of modifying or removing this file,
+# create a /usr/local/etc/pkg/repos/FreeBSD.conf file:
+#
+#   mkdir -p /usr/local/etc/pkg/repos
+#   echo "FreeBSD: { enabled: no }" > /usr/local/etc/pkg/repos/FreeBSD.conf
+#
+
+FreeBSD: {
+  url: "pkg+http://pkg.FreeBSD.org/${ABI}/latest;,
+  mirror_type: "srv",
+  signature_type: "fingerprints",
+  fingerprints: "/usr/share/keys/pkg",
+  enabled: yes
+}

Copied and modified: head/usr.sbin/pkg/FreeBSD.conf.quarterly (from r346627, 
head/usr.sbin/pkg/FreeBSD.conf)
==
--- head/usr.sbin/pkg/FreeBSD.conf  Wed Apr 24 05:52:24 2019
(r346627, copy source)
+++ head/usr.sbin/pkg/FreeBSD.conf.quarterlyWed Apr 24 06:25:21 2019
(r346628)
@@ -8,7 +8,7 @@
 #
 
 FreeBSD: {
-  url: "pkg+http://pkg.FreeBSD.org/${ABI}/latest;,
+  url: "pkg+http://pkg.FreeBSD.org/${ABI}/quarterly;,
   mirror_type: "srv",
   signature_type: "fingerprints",
   fingerprints: "/usr/share/keys/pkg",

Modified: head/usr.sbin/pkg/Makefile
==
--- head/usr.sbin/pkg/Makefile  Wed Apr 24 05:52:24 2019(r346627)
+++ head/usr.sbin/pkg/Makefile  Wed Apr 24 06:25:21 2019(r346628)
@@ -1,6 +1,8 @@
 # $FreeBSD$
 
-CONFS= FreeBSD.conf
+PKGCONFBRANCH?=latest
+CONFS= FreeBSD.conf.${PKGCONFBRANCH}
+CONFSNAME= FreeBSD.conf
 CONFSDIR=  /etc/pkg
 CONFSMODE= 644
 PROG=  pkg
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r345858 - in head/release: . tools

[Colin Percival]

Author: cperciva
Date: Wed Apr  3 21:54:47 2019
New Revision: 345858
URL: https://svnweb.freebsd.org/changeset/base/345858

Log:
  Add support for cross-building cloudware images.
  
  If MACHINE_ARCH doesn't match TARGET_ARCH, and we're not in the special
  case of building i386 images on an amd64 host, we need to pull in the
  qemu-user-static package; this allows us to run some commands inside
  the VM disk image chroot, most notably to install packages.
  
  Reviewed by:  gjb
  MFC after:2 weeks
  Sponsored by: FreeBSD/EC2 patreon (https://www.patreon.com/cperciva)

Modified:
  head/release/Makefile.vm
  head/release/tools/ec2.conf
  head/release/tools/vmimage.subr

Modified: head/release/Makefile.vm
==
--- head/release/Makefile.vmWed Apr  3 21:48:09 2019(r345857)
+++ head/release/Makefile.vmWed Apr  3 21:54:47 2019(r345858)
@@ -39,6 +39,24 @@ VAGRANT-VMWARE_FORMAT=   vmdk
 VAGRANT-VMWARE_DESC=   Vagrant Image for VMWare
 VAGRANT-VMWARE_DISK=   ${OSRELEASE}.vmware.${VAGRANT_FORMAT}
 
+emulator-portinstall:
+.if ${TARGET_ARCH} != ${MACHINE_ARCH}
+.if ( ${TARGET_ARCH} != "i386" ) || ( ${MACHINE_ARCH} != "amd64" )
+.if !exists(/usr/local/bin/qemu-${TARGET_ARCH}-static)
+.if exists(${PORTSDIR}/emulators/qemu-user-static/Makefile)
+   env - PATH=$$PATH make -C ${PORTSDIR}/emulators/qemu-user-static 
BATCH=1 all install clean
+.else
+.if !exists(/usr/local/sbin/pkg-static)
+   env ASSUME_ALWAYS_YES=yes pkg bootstrap -y
+.endif
+   env ASSUME_ALWAYS_YES=yes pkg install -y emulators/qemu-user-static
+.endif
+.endif
+
+QEMUSTATIC=/usr/local/bin/qemu-${TARGET_ARCH}-static
+.endif
+.endif
+
 .if defined(WITH_CLOUDWARE) && !empty(WITH_CLOUDWARE) && !empty(CLOUDWARE)
 . for _CW in ${CLOUDWARE}
 CLOUDTARGETS+= cw-${_CW:tl}
@@ -53,9 +71,10 @@ ${_CW:tu}IMAGE=  ${_CW:tl}.${${_CW:tu}_FORMAT}
 ${_CW:tu}CONF?=${.CURDIR}/tools/${_CW:tl}.conf
 .  endif
 
-cw-${_CW:tl}:
+cw-${_CW:tl}: emulator-portinstall
mkdir -p ${.OBJDIR}/${.TARGET}
env TARGET=${TARGET} TARGET_ARCH=${TARGET_ARCH} SWAPSIZE=${SWAPSIZE} \
+   QEMUSTATIC=${QEMUSTATIC} \
${.CURDIR}/scripts/mk-vmimage.sh \
-C ${.CURDIR}/tools/vmimage.subr -d ${.OBJDIR}/${.TARGET} \
-i ${.OBJDIR}/${_CW:tl}.img -s ${VMSIZE} -f ${${_CW:tu}_FORMAT} 
\

Modified: head/release/tools/ec2.conf
==
--- head/release/tools/ec2.conf Wed Apr  3 21:48:09 2019(r345857)
+++ head/release/tools/ec2.conf Wed Apr  3 21:54:47 2019(r345858)
@@ -40,7 +40,7 @@ vm_extra_pre_umount() {
# catalogue and install or update pkg when the instance first
# launches, so these files would just be replaced anyway; removing
# them from the image allows it to boot faster.
-   chroot ${DESTDIR} env ASSUME_ALWAYS_YES=yes \
+   chroot ${DESTDIR} ${EMULATOR} env ASSUME_ALWAYS_YES=yes \
/usr/sbin/pkg delete -f -y pkg
rm ${DESTDIR}/var/db/pkg/repo-*.sqlite
 

Modified: head/release/tools/vmimage.subr
==
--- head/release/tools/vmimage.subr Wed Apr  3 21:48:09 2019
(r345857)
+++ head/release/tools/vmimage.subr Wed Apr  3 21:54:47 2019
(r345858)
@@ -148,10 +148,15 @@ vm_install_base() {
hostname="$(echo $(uname -o) | tr '[:upper:]' '[:lower:]')"
echo "hostname=\"${hostname}\"" >> ${DESTDIR}/etc/rc.conf
 
+   if ! [ -z "${QEMUSTATIC}" ]; then
+   export EMULATOR=/qemu
+   cp ${QEMUSTATIC} ${DESTDIR}/${EMULATOR}
+   fi
+
mkdir -p ${DESTDIR}/dev
mount -t devfs devfs ${DESTDIR}/dev
-   chroot ${DESTDIR} /usr/bin/newaliases
-   chroot ${DESTDIR} /etc/rc.d/ldconfig forcestart
+   chroot ${DESTDIR} ${EMULATOR} /usr/bin/newaliases
+   chroot ${DESTDIR} ${EMULATOR} /bin/sh /etc/rc.d/ldconfig forcestart
umount_loop ${DESTDIR}/dev
 
cp /etc/resolv.conf ${DESTDIR}/etc/resolv.conf
@@ -188,9 +193,9 @@ vm_extra_install_packages() {
fi
mkdir -p ${DESTDIR}/dev
mount -t devfs devfs ${DESTDIR}/dev
-   chroot ${DESTDIR} env ASSUME_ALWAYS_YES=yes \
+   chroot ${DESTDIR} ${EMULATOR} env ASSUME_ALWAYS_YES=yes \
/usr/sbin/pkg bootstrap -y
-   chroot ${DESTDIR} env ASSUME_ALWAYS_YES=yes \
+   chroot ${DESTDIR} ${EMULATOR} env ASSUME_ALWAYS_YES=yes \
/usr/sbin/pkg install -y ${VM_EXTRA_PACKAGES}
umount_loop ${DESTDIR}/dev
 
@@ -210,13 +215,16 @@ vm_extra_pre_umount() {
# Note: When overriding this function, removing resolv.conf in the
# disk image must be included.
 
+   if ! [ -z "${QEMUSTATIC}" ]; then
+   rm -f ${DESTDIR}/${EMULATOR}
+   fi
rm -f ${DESTDIR}/etc/resolv.conf

svn commit: r345408 - head/sys/arm64/conf

[Colin Percival]

Author: cperciva
Date: Fri Mar 22 06:36:40 2019
New Revision: 345408
URL: https://svnweb.freebsd.org/changeset/base/345408

Log:
  Add nvme support to the arm64 GENERIC kernel.
  
  Submitted by: Greg V
  Differential Revision:https://reviews.freebsd.org/D19657

Modified:
  head/sys/arm64/conf/GENERIC

Modified: head/sys/arm64/conf/GENERIC
==
--- head/sys/arm64/conf/GENERIC Fri Mar 22 06:33:26 2019(r345407)
+++ head/sys/arm64/conf/GENERIC Fri Mar 22 06:36:40 2019(r345408)
@@ -171,6 +171,11 @@ device da
 # ATA/SCSI peripherals
 device pass# Passthrough device (direct ATA/SCSI access)
 
+# NVM Express (NVMe) support
+device nvme# base NVMe driver
+optionsNVME_USE_NVD=0  # prefer the cam(4) based nda(4) driver
+device nvd # expose NVMe namespaces as disks, depends on 
nvme
+
 # MMC/SD/SDIO Card slot support
 device sdhci
 device sdhci_xenon # Marvell Xenon SD/MMC controller
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r345407 - head/sys/modules

[Colin Percival]

Author: cperciva
Date: Fri Mar 22 06:33:26 2019
New Revision: 345407
URL: https://svnweb.freebsd.org/changeset/base/345407

Log:
  Build if_ena.ko on arm64.
  
  This module provides support for the Amazon Elastic Network Adapter; it
  was previously only built on x86 architectures, but Amazon EC2 now also
  has ARM64 instances with this hardware.
  
  Submitted by: Greg V

Modified:
  head/sys/modules/Makefile

Modified: head/sys/modules/Makefile
==
--- head/sys/modules/Makefile   Fri Mar 22 06:28:37 2019(r345406)
+++ head/sys/modules/Makefile   Fri Mar 22 06:33:26 2019(r345407)
@@ -482,6 +482,7 @@ SUBDIR+=fdt
 ${MACHINE_CPUARCH} == "i386"
 SUBDIR+=   linprocfs
 SUBDIR+=   linsysfs
+_ena=  ena
 .endif
 
 .if ${MK_NAND} != "no" || defined(ALL_MODULES)
@@ -587,7 +588,6 @@ _cs=cs
 _dpms= dpms
 _ed=   ed
 _em=   em
-_ena=  ena
 _ep=   ep
 _et=   et
 _exca= exca
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r345406 - head/sys/dev/uart

[Colin Percival]

Author: cperciva
Date: Fri Mar 22 06:28:37 2019
New Revision: 345406
URL: https://svnweb.freebsd.org/changeset/base/345406

Log:
  Initialize uart_bus_space_mem.
  
  This value was being used uninitialized, resulting in predictable issues
  on systems with memory-mapped UART registers.
  
  A case could be made that memmap_bus should be declared in a header
  rather than being declared in each .c file which needs to refer to it,
  but that's a broader style question.
  
  This commit unbreaks hw.uart.console="mm:..." on ARM64.
  
  Submitted by: Greg V

Modified:
  head/sys/dev/uart/uart_cpu_arm64.c

Modified: head/sys/dev/uart/uart_cpu_arm64.c
==
--- head/sys/dev/uart/uart_cpu_arm64.c  Fri Mar 22 06:21:03 2019
(r345405)
+++ head/sys/dev/uart/uart_cpu_arm64.c  Fri Mar 22 06:28:37 2019
(r345406)
@@ -65,8 +65,9 @@ __FBSDID("$FreeBSD$");
 /*
  * UART console routines.
  */
+extern struct bus_space memmap_bus;
 bus_space_tag_t uart_bus_space_io;
-bus_space_tag_t uart_bus_space_mem;
+bus_space_tag_t uart_bus_space_mem = _bus;
 
 int
 uart_cpu_eqres(struct uart_bas *b1, struct uart_bas *b2)
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r345405 - head/sys/dev/uart

[Colin Percival]

Author: cperciva
Date: Fri Mar 22 06:21:03 2019
New Revision: 345405
URL: https://svnweb.freebsd.org/changeset/base/345405

Log:
  Obey SPCR AccessWidth parameter.
  
  The "access width" value was hard-coded as 2, indicating 32-bit accesses;
  instead, use the value specified in the SPCR table.
  
  This unbreaks the console on EC2 "A1" family instances.
  
  Submitted by: Greg V

Modified:
  head/sys/dev/uart/uart_cpu_arm64.c

Modified: head/sys/dev/uart/uart_cpu_arm64.c
==
--- head/sys/dev/uart/uart_cpu_arm64.c  Fri Mar 22 06:02:06 2019
(r345404)
+++ head/sys/dev/uart/uart_cpu_arm64.c  Fri Mar 22 06:21:03 2019
(r345405)
@@ -149,7 +149,7 @@ uart_cpu_acpi_probe(struct uart_class **classp, bus_sp
 
*classp = cd->cd_class;
*rclk = 0;
-   *shiftp = 2;
+   *shiftp = spcr->SerialPort.AccessWidth - 1;
*iowidthp = spcr->SerialPort.BitWidth / 8;
 
 out:
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r345369 - head/sys/dev/uart

[Colin Percival]

Author: cperciva
Date: Thu Mar 21 08:54:34 2019
New Revision: 345369
URL: https://svnweb.freebsd.org/changeset/base/345369

Log:
  Recognize the Amazon PCI serial device found in a1.* EC2 instances
  as an NS8250 UART.
  
  This is the same as the UART found in EC2 "bare metal" instances,
  except that the card vendor shows up as 0x rather than 0x1d0f.
  This seems like a bug in the EC2 firmware; but we might as well support
  it anyway.
  
  Reported by:  Greg V

Modified:
  head/sys/dev/uart/uart_bus_pci.c

Modified: head/sys/dev/uart/uart_bus_pci.c
==
--- head/sys/dev/uart/uart_bus_pci.cThu Mar 21 08:51:32 2019
(r345368)
+++ head/sys/dev/uart/uart_bus_pci.cThu Mar 21 08:54:34 2019
(r345369)
@@ -125,6 +125,7 @@ static const struct pci_id pci_ns8250_ids[] = {
128 * DEFAULT_RCLK, 2},
 { 0x14e4, 0x4344, 0x, 0, "Sony Ericsson GC89 PC Card", 0x10},
 { 0x151f, 0x, 0x, 0, "TOPIC Semiconductor TP560 56k modem", 0x10 },
+{ 0x1d0f, 0x8250, 0x, 0, "Amazon PCI serial device", 0x10 },
 { 0x1d0f, 0x8250, 0x1d0f, 0, "Amazon PCI serial device", 0x10 },
 { 0x1fd4, 0x1999, 0x1fd4, 0x0001, "Sunix SER5 Serial Port", 0x10,
8 * DEFAULT_RCLK },
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r345317 - head/release/tools

[Colin Percival]

Author: cperciva
Date: Wed Mar 20 07:24:21 2019
New Revision: 345317
URL: https://svnweb.freebsd.org/changeset/base/345317

Log:
  Only install amazon-ssm-agent into amd64 AMIs.
  
  This package does not exist on aarch64 at present.

Modified:
  head/release/tools/ec2.conf

Modified: head/release/tools/ec2.conf
==
--- head/release/tools/ec2.conf Wed Mar 20 07:19:09 2019(r345316)
+++ head/release/tools/ec2.conf Wed Mar 20 07:24:21 2019(r345317)
@@ -6,7 +6,15 @@
 # Packages to install into the image we're creating.  This is a deliberately
 # minimalist set, providing only the packages necessary to bootstrap further
 # package installation as specified via EC2 user-data.
-export VM_EXTRA_PACKAGES="ec2-scripts firstboot-freebsd-update firstboot-pkgs 
dual-dhclient amazon-ssm-agent"
+export VM_EXTRA_PACKAGES="ec2-scripts firstboot-freebsd-update firstboot-pkgs 
dual-dhclient"
+
+# Include the amazon-ssm-agent package in amd64 images, since some users want
+# to be able to use it on systems which are not connected to the Internet.
+# (It is not enabled by default, however.)  This package does not exist for
+# aarch64, so we have to be selective about when we install it.
+if [ "${TARGET_ARCH}" = "amd64" ]; then
+   export VM_EXTRA_PACKAGES="${VM_EXTRA_PACKAGES} amazon-ssm-agent"
+fi
 
 # Set to a list of third-party software to enable in rc.conf(5).
 export VM_RC_LIST="ec2_configinit ec2_fetchkey ec2_loghostkey 
firstboot_freebsd_update firstboot_pkgs ntpd"
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r345316 - head/release

[Colin Percival]

Author: cperciva
Date: Wed Mar 20 07:19:09 2019
New Revision: 345316
URL: https://svnweb.freebsd.org/changeset/base/345316

Log:
  Pass --arm64 to bsdec2-image-upload when building ARM64 AMIs.
  
  Future commits will allow the resulting EC2 AMIs to actually boot and
  be usable.

Modified:
  head/release/Makefile.ec2

Modified: head/release/Makefile.ec2
==
--- head/release/Makefile.ec2   Wed Mar 20 03:50:45 2019(r345315)
+++ head/release/Makefile.ec2   Wed Mar 20 07:19:09 2019(r345316)
@@ -42,6 +42,9 @@ PUBLICSNAP=   --publicsnap
 EC2SNSREL= ${REVISION}-${BRANCH}
 EC2SNSVERS=${EC2_SVNBRANCH}@${EC2_SVNREV}
 .endif
+.if ${TARGET_ARCH} != "amd64"
+EC2ARCH=   --${TARGET_ARCH:S/aarch64/arm64/}
+.endif
 
 CLEANFILES+=   ec2ami
 
@@ -82,7 +85,8 @@ ec2ami: cw-ec2 ${CW_EC2_PORTINSTALL}
@echo "--"
@false
 .endif
-   /usr/local/bin/bsdec2-image-upload ${PUBLISH} ${PUBLICSNAP} --sriov 
--ena \
+   /usr/local/bin/bsdec2-image-upload ${PUBLISH} ${PUBLICSNAP} \
+   ${EC2ARCH} --sriov --ena \
${.OBJDIR}/ec2.raw \
"${TYPE} ${REVISION}-${BRANCH}-${TARGET}${AMINAMESUFFIX}" \
"${TYPE}/${TARGET} ${EC2_SVNBRANCH}@${EC2_SVNREV}" \
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r332100 - in head: . lib/libc/gen sys/sys

[Colin Percival]

On 4/6/18 6:00 AM, Ed Schouten wrote:
> Author: ed
> Date: Fri Apr  6 13:00:45 2018
> New Revision: 332100
> URL: https://svnweb.freebsd.org/changeset/base/332100
> 
> Log:
>   Let syslog(3) use RFC 5424.
>   
>   With r332099 changing syslogd(8) to parse RFC 5424 formatted syslog
>   messages, go ahead and also change the syslog(3) libc function to
>   generate them. Compared to RFC 3164, RFC 5424 has various advantages,
>   such as sub-second precision for log entry timestamps.
>   
>   As this change could have adverse effects when not updating syslogd(8)
>   or using a different system logging daemon, add a notice to UPDATING and
>   increase __FreeBSD_version.
>   
>   Differential Revision:  https://reviews.freebsd.org/D14926

It looks like this changes the format in which messages are written to
stderr via the LOG_PERROR flag; this is visible if you run
# echo foo | logger -s
which used to print "root: foo" but now prints "root 40038 - - foo".

Was this intentional?

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r344315 - head/release/tools

[Colin Percival]

Author: cperciva
Date: Tue Feb 19 23:24:39 2019
New Revision: 344315
URL: https://svnweb.freebsd.org/changeset/base/344315

Log:
  Fix sed script to insert Amazon NTP server into ntp.conf once rather
  than twice.
  
  Reported by:  Rafal Lukawiecki
  MFC after:1 week

Modified:
  head/release/tools/ec2.conf

Modified: head/release/tools/ec2.conf
==
--- head/release/tools/ec2.conf Tue Feb 19 23:22:39 2019(r344314)
+++ head/release/tools/ec2.conf Tue Feb 19 23:24:39 2019(r344315)
@@ -96,7 +96,7 @@ vm_extra_pre_umount() {
 
# Use the NTP service provided by Amazon
sed -i '' -e 's/^pool/#pool/' \
-   -e 's/^#server.*/server 169.254.169.123 iburst/' \
+   -e '1,/^#server/s/^#server.*/server 169.254.169.123 iburst/' \
${DESTDIR}/etc/ntp.conf
 
# The first time the AMI boots, the installed "first boot" scripts
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r343918 - head/libexec/rc/rc.d

[Colin Percival]

Author: cperciva
Date: Fri Feb  8 19:19:31 2019
New Revision: 343918
URL: https://svnweb.freebsd.org/changeset/base/343918

Log:
  Teach /etc/rc.d/growfs how to handle systems running ZFS.
  
  There are many cases which this code does not handle (e.g. ZFS mirrors)
  but the code can handle the single-disk case -- so it's enough to take
  care of the "disk image which gets booted into a VM with a larger than
  expected disk" case for which this firstboot script was created.
  
  MFC after:1 week
  Differential Revision:https://reviews.freebsd.org/D19095

Modified:
  head/libexec/rc/rc.d/growfs

Modified: head/libexec/rc/rc.d/growfs
==
--- head/libexec/rc/rc.d/growfs Fri Feb  8 18:31:54 2019(r343917)
+++ head/libexec/rc/rc.d/growfs Fri Feb  8 19:19:31 2019(r343918)
@@ -49,7 +49,20 @@ rcvar="growfs_enable"
 growfs_start ()
 {
echo "Growing root partition to fill device"
-   rootdev=$(df / | tail -n 1 | awk '{ sub("/dev/", "", $1); print $1 }')
+   FSTYPE=$(mount -p | awk '{ if ( $2 == "/") { print $3 }}')
+   FSDEV=$(mount -p | awk '{ if ( $2 == "/") { print $1 }}')
+   case "$FSTYPE" in
+   ufs)
+   rootdev=${FSDEV#/dev/}
+   ;;
+   zfs)
+   pool=${FSDEV%%/*}
+   rootdev=$(zpool list -v $pool | tail -n 1 | awk '{ print $1 }')
+   ;;
+   *)
+   echo "Don't know how to grow root filesystem type: $FSTYPE"
+   return
+   esac
if [ x"$rootdev" = x"${rootdev%/*}" ]; then
# raw device
rawdev="$rootdev"
@@ -91,7 +104,14 @@ growfs_start ()
}
 }' dev="$rawdev"
gpart commit "$rootdev"
-   growfs -y /dev/"$rootdev"
+   case "$FSTYPE" in
+   ufs)
+   growfs -y /dev/"$rootdev"
+   ;;
+   zfs)
+   zpool online -e $pool $rootdev
+   ;;
+   esac
 }
 
 load_rc_config $name
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r343480 - head/lib/libfigpar

[Colin Percival]

On 1/26/19 1:36 PM, Rodney W. Grimes wrote:
>> Author: se
>> Date: Sat Jan 26 21:30:26 2019
>> New Revision: 343480
>> URL: https://svnweb.freebsd.org/changeset/base/343480
>>
>> Log:
>>   Silence Clang Scan warning about potentially unsafe use of strcpy.
>>   
>>   While this is a false positive, the use of strdup() simplifies the code.
> 
> Though that might be true, it also has to recalculate the
> length of the string which was already known by slen.
> 
> I am not sure how often this code is called,
> but that is wasted cycles in a library.

The length of the string was already being recalculated, by strcpy.

It seems to me that this could be written as

temp = malloc(slen + 1);
if (temp == NULL) /* could not allocate memory */
return (-1);
memcpy(temp, source, slen + 1);

which avoids both recalculating the string length and using strcpy?

>> Modified: head/lib/libfigpar/string_m.c
>> ==
>> --- head/lib/libfigpar/string_m.cSat Jan 26 20:43:28 2019
>> (r343479)
>> +++ head/lib/libfigpar/string_m.cSat Jan 26 21:30:26 2019
>> (r343480)
>> @@ -119,10 +119,9 @@ replaceall(char *source, const char *find, const char 
>>  
>>  /* If replace is longer than find, we'll need to create a temp copy */
>>  if (rlen > flen) {
>> -temp = malloc(slen + 1);
>> -if (errno != 0) /* could not allocate memory */
>> +temp = strdup(source);
>> +if (temp == NULL) /* could not allocate memory */
>>  return (-1);
>> -strcpy(temp, source);
>>  } else
>>  temp = source;

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r342875 - head/release/tools

[Colin Percival]

Author: cperciva
Date: Wed Jan  9 03:55:25 2019
New Revision: 342875
URL: https://svnweb.freebsd.org/changeset/base/342875

Log:
  Turn off ec2_ephemeralswap for now
  
  This script broke around FreeBSD 11.0 as a result of SWAPMETA no longer
  being reported by vmstat -z; but it also needs to be reworked due to the
  arrival in EC2 of nvme ephemeral disks.
  
  I'll turn this option back on after I've found time to rewrite the
  script in question.
  
  PR:   234686
  Reported by:  meta@
  MFC after:1 week

Modified:
  head/release/tools/ec2.conf

Modified: head/release/tools/ec2.conf
==
--- head/release/tools/ec2.conf Wed Jan  9 02:47:07 2019(r342874)
+++ head/release/tools/ec2.conf Wed Jan  9 03:55:25 2019(r342875)
@@ -9,7 +9,7 @@
 export VM_EXTRA_PACKAGES="ec2-scripts firstboot-freebsd-update firstboot-pkgs 
dual-dhclient amazon-ssm-agent"
 
 # Set to a list of third-party software to enable in rc.conf(5).
-export VM_RC_LIST="ec2_configinit ec2_fetchkey ec2_ephemeralswap 
ec2_loghostkey firstboot_freebsd_update firstboot_pkgs ntpd"
+export VM_RC_LIST="ec2_configinit ec2_fetchkey ec2_loghostkey 
firstboot_freebsd_update firstboot_pkgs ntpd"
 
 # Build with a 3 GB UFS partition; the growfs rc.d script will expand
 # the partition to fill the root disk after the EC2 instance is launched.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r338921 - head/sys/dev/uart

[Colin Percival]

Author: cperciva
Date: Mon Sep 24 22:15:04 2018
New Revision: 338921
URL: https://svnweb.freebsd.org/changeset/base/338921

Log:
  Recognize the Amazon PCI serial device found in i3.metal EC2 instances
  as an NS8250 UART.
  
  Reviewed by:  sbruno, imp
  Approved by:  re (delphij)
  Sponsored by: https://www.patreon.com/cperciva
  Differential Revision:https://reviews.freebsd.org/D17250

Modified:
  head/sys/dev/uart/uart_bus_pci.c

Modified: head/sys/dev/uart/uart_bus_pci.c
==
--- head/sys/dev/uart/uart_bus_pci.cMon Sep 24 20:46:45 2018
(r338920)
+++ head/sys/dev/uart/uart_bus_pci.cMon Sep 24 22:15:04 2018
(r338921)
@@ -125,6 +125,7 @@ static const struct pci_id pci_ns8250_ids[] = {
128 * DEFAULT_RCLK, 2},
 { 0x14e4, 0x4344, 0x, 0, "Sony Ericsson GC89 PC Card", 0x10},
 { 0x151f, 0x, 0x, 0, "TOPIC Semiconductor TP560 56k modem", 0x10 },
+{ 0x1d0f, 0x8250, 0x1d0f, 0, "Amazon PCI serial device", 0x10 },
 { 0x1fd4, 0x1999, 0x1fd4, 0x0001, "Sunix SER5 Serial Port", 0x10,
8 * DEFAULT_RCLK },
 { 0x8086, 0x0f0a, 0x, 0, "Intel ValleyView LPIO1 HSUART#1", 0x10,
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r335668 - head/sys/x86/xen

[Colin Percival]

This breaks the kernel boot in EC2 Xen-based instances; it now hangs
in the taskqgroup_adjust_if_io_tqg SYSINIT.  I've attached a verbose
dmesg from r335666 in case that helps you figure out what's going on.

My guess is that EC2 is numbering virtual CPUs oddly -- but regardless
of the cause, this definitely needs to be fixed.

Colin Percival

On 6/26/18 8:00 AM, Roger Pau Monné wrote:
> Author: royger
> Date: Tue Jun 26 15:00:54 2018
> New Revision: 335668
> URL: https://svnweb.freebsd.org/changeset/base/335668
> 
> Log:
>   xen: obtain vCPU ID from CPUID
>   
>   The Xen vCPU ID can be fetched from the cpuid instead of inferring it
>   from the ACPI ID.
>   
>   Sponsored by: Citrix Systems R
> 
> Modified:
>   head/sys/x86/xen/hvm.c
> 
> Modified: head/sys/x86/xen/hvm.c
> ==
> --- head/sys/x86/xen/hvm.cTue Jun 26 14:48:23 2018(r335667)
> +++ head/sys/x86/xen/hvm.cTue Jun 26 15:00:54 2018(r335668)
> @@ -56,6 +56,7 @@ __FBSDID("$FreeBSD$");
>  #include 
>  #include 
>  
> +#include 
>  #include 
>  #include 
>  
> @@ -103,6 +104,9 @@ TUNABLE_INT("hw.xen.disable_pv_disks", _disable_pv
>  TUNABLE_INT("hw.xen.disable_pv_nics", _disable_pv_nics);
>  
>  /*-- XEN Hypervisor Probe and Setup 
> --*/
> +
> +static uint32_t cpuid_base;
> +
>  static uint32_t
>  xen_hvm_cpuid_base(void)
>  {
> @@ -123,21 +127,21 @@ xen_hvm_cpuid_base(void)
>  static int
>  xen_hvm_init_hypercall_stubs(enum xen_hvm_init_type init_type)
>  {
> - uint32_t base, regs[4];
> + uint32_t regs[4];
>  
>   if (xen_pv_domain()) {
>   /* hypercall page is already set in the PV case */
>   return (0);
>   }
>  
> - base = xen_hvm_cpuid_base();
> - if (base == 0)
> + cpuid_base = xen_hvm_cpuid_base();
> + if (cpuid_base == 0)
>   return (ENXIO);
>  
>   if (init_type == XEN_HVM_INIT_COLD) {
>   int major, minor;
>  
> - do_cpuid(base + 1, regs);
> + do_cpuid(cpuid_base + 1, regs);
>  
>   major = regs[0] >> 16;
>   minor = regs[0] & 0x;
> @@ -165,7 +169,7 @@ xen_hvm_init_hypercall_stubs(enum xen_hvm_init_type in
>   /*
>* Find the hypercall pages.
>*/
> - do_cpuid(base + 2, regs);
> + do_cpuid(cpuid_base + 2, regs);
>   if (regs[0] != 1)
>   return (EINVAL);
>  
> @@ -371,31 +375,14 @@ xen_hvm_sysinit(void *arg __unused)
>  {
>   xen_hvm_init(XEN_HVM_INIT_COLD);
>  }
> +SYSINIT(xen_hvm_init, SI_SUB_HYPERVISOR, SI_ORDER_FIRST, xen_hvm_sysinit, 
> NULL);
>  
>  static void
> -xen_set_vcpu_id(void)
> -{
> - struct pcpu *pc;
> - int i;
> -
> - if (!xen_hvm_domain())
> - return;
> -
> - /* Set vcpu_id to acpi_id */
> - CPU_FOREACH(i) {
> - pc = pcpu_find(i);
> - pc->pc_vcpu_id = pc->pc_acpi_id;
> - if (bootverbose)
> - printf("XEN: CPU %u has VCPU ID %u\n",
> -i, pc->pc_vcpu_id);
> - }
> -}
> -
> -static void
>  xen_hvm_cpu_init(void)
>  {
>   struct vcpu_register_vcpu_info info;
>   struct vcpu_info *vcpu_info;
> + uint32_t regs[4];
>   int cpu, rc;
>  
>   if (!xen_domain())
> @@ -410,6 +397,22 @@ xen_hvm_cpu_init(void)
>   return;
>   }
>  
> + /*
> +  * Set vCPU ID. If available fetch the ID from CPUID, if not just use
> +  * the ACPI ID.
> +  */
> + KASSERT(cpuid_base != 0, ("Invalid base Xen CPUID leaf"));
> + cpuid_count(cpuid_base + 4, 0, regs);
> + PCPU_SET(vcpu_id, (regs[0] & XEN_HVM_CPUID_VCPU_ID_PRESENT) ?
> + regs[1] : PCPU_GET(acpi_id));
> +
> + /*
> +  * Set the vCPU info.
> +  *
> +  * NB: the vCPU info for vCPUs < 32 can be fetched from the shared info
> +  * page, but in order to make sure the mapping code is correct always
> +  * attempt to map the vCPU info at a custom place.
> +  */
>   vcpu_info = DPCPU_PTR(vcpu_local_info);
>   cpu = PCPU_GET(vcpu_id);
>   info.mfn = vtophys(vcpu_info) >> PAGE_SHIFT;
> @@ -421,7 +424,4 @@ xen_hvm_cpu_init(void)
>   else
>   DPCPU_SET(vcpu_info, vcpu_info);
>  }
> -
> -SYSINIT(xen_hvm_init, SI_SUB_HYPERVISOR, SI_ORDER_FIRST, xen_hvm_sysinit, 
> NULL);
>  SYSINIT(xen_hvm_cpu_init, SI_SUB_INTR, SI_ORDER_FIRST, xen_hvm_cpu_init, 
> NULL);
> -SYSINIT(xen_set_vcpu_id, SI_SUB_CPU, SI_ORDER_ANY, xen_set_vcpu_id, NULL);
> 
> 
> 

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r338321 - head/release/tools

[Colin Percival]

Author: cperciva
Date: Sun Aug 26 03:56:54 2018
New Revision: 338321
URL: https://svnweb.freebsd.org/changeset/base/338321

Log:
  Disable atkbd0 and atkdbc0 in EC2 AMIs.  This has the effect of skipping
  the probing and attaching of the PS/2 mouse (not present on EC2) and
  keyboard (emulated, but not accessible via EC2).
  
  Note that we disable atkbd0 separately even though during device probing
  it shows up as a child of atkbdc0; this is necessary because the device
  is also initialized during the early console setup from hammer_time.
  
  This change cuts the kernel boot time on an EC2 c5.4xlarge instance from
  7259ms down to 4727 ms.
  
  Approved by:  re (marius)

Modified:
  head/release/tools/ec2.conf

Modified: head/release/tools/ec2.conf
==
--- head/release/tools/ec2.conf Sun Aug 26 02:09:20 2018(r338320)
+++ head/release/tools/ec2.conf Sun Aug 26 03:56:54 2018(r338321)
@@ -68,6 +68,13 @@ vm_extra_pre_umount() {
echo 'autoboot_delay="-1"' >> ${DESTDIR}/boot/loader.conf
echo 'beastie_disable="YES"' >> ${DESTDIR}/boot/loader.conf
 
+   # The emulated keyboard attached to EC2 instances is inaccessible to
+   # users, and there is no mouse attached at all; disable to keyboard
+   # and the keyboard controller (to which the mouse would attach, if
+   # one existed) in order to save time in device probing.
+   echo 'hint.atkbd.0.disabled=1' >> ${DESTDIR}/boot/loader.conf
+   echo 'hint.atkbdc.0.disabled=1' >> ${DESTDIR}/boot/loader.conf
+
# EC2 has two consoles: An emulated serial port ("system log"),
# which has been present since 2006; and a VGA console ("instance
# screenshot") which was introduced in 2016.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r338316 - in head/sys/dev/vt: . hw/efifb hw/fb hw/vga

[Colin Percival]

Author: cperciva
Date: Sat Aug 25 16:14:56 2018
New Revision: 338316
URL: https://svnweb.freebsd.org/changeset/base/338316

Log:
  Speed up vt(4) by keeping a record of the most recently drawn character and
  the foreground and background colours.  In bitblt_text functions, compare
  values to this cache and don't re-draw the characters if they haven't changed.
  When invalidating the display, clear this cache in order to force characters
  to be redrawn; also force full redraws between suspend/resume pairs since odd
  artifacts can otherwise result.
  
  When scrolling the display (which is where most time is spent within the vt
  driver) this yields a significant performance improvement if most lines are
  less than the width of the terminal, since this avoids re-drawing blanks on
  top of blanks.
  
  (Note that "re-drawing" here includes writing to the VGA text mode buffer; on
  virtualized systems this can be extremely slow since it triggers a glyph
  being rendered onto a 640x480 screen).
  
  On a c5.4xlarge EC2 instance (with emulated text mode VGA) this cuts the time
  spent in vt(4) during the kernel boot from 1200 ms to 700ms; on my laptop
  (with a 3200x1800 display) the corresponding time is reduced from 970 ms down
  to 155 ms.
  
  Reviewed by:  imp, cem
  Approved by:  re (gjb)
  Relnotes: Significant speedup in vt(4) and the system boot generally.
  Differential Revision:https://reviews.freebsd.org/D16723

Modified:
  head/sys/dev/vt/hw/efifb/efifb.c
  head/sys/dev/vt/hw/fb/vt_early_fb.c
  head/sys/dev/vt/hw/fb/vt_fb.c
  head/sys/dev/vt/hw/fb/vt_fb.h
  head/sys/dev/vt/hw/vga/vt_vga.c
  head/sys/dev/vt/vt.h
  head/sys/dev/vt/vt_core.c

Modified: head/sys/dev/vt/hw/efifb/efifb.c
==
--- head/sys/dev/vt/hw/efifb/efifb.cSat Aug 25 15:59:51 2018
(r338315)
+++ head/sys/dev/vt/hw/efifb/efifb.cSat Aug 25 16:14:56 2018
(r338316)
@@ -58,6 +58,7 @@ static struct vt_driver vt_efifb_driver = {
.vd_init = vt_efifb_init,
.vd_blank = vt_fb_blank,
.vd_bitblt_text = vt_fb_bitblt_text,
+   .vd_invalidate_text = vt_fb_invalidate_text,
.vd_bitblt_bmp = vt_fb_bitblt_bitmap,
.vd_drawrect = vt_fb_drawrect,
.vd_setpixel = vt_fb_setpixel,

Modified: head/sys/dev/vt/hw/fb/vt_early_fb.c
==
--- head/sys/dev/vt/hw/fb/vt_early_fb.c Sat Aug 25 15:59:51 2018
(r338315)
+++ head/sys/dev/vt/hw/fb/vt_early_fb.c Sat Aug 25 16:14:56 2018
(r338316)
@@ -60,6 +60,7 @@ static struct vt_driver vt_fb_early_driver = {
.vd_init = vt_efb_init,
.vd_blank = vt_fb_blank,
.vd_bitblt_text = vt_fb_bitblt_text,
+   .vd_invalidate_text = vt_fb_invalidate_text,
.vd_bitblt_bmp = vt_fb_bitblt_bitmap,
.vd_drawrect = vt_fb_drawrect,
.vd_setpixel = vt_fb_setpixel,

Modified: head/sys/dev/vt/hw/fb/vt_fb.c
==
--- head/sys/dev/vt/hw/fb/vt_fb.c   Sat Aug 25 15:59:51 2018
(r338315)
+++ head/sys/dev/vt/hw/fb/vt_fb.c   Sat Aug 25 16:14:56 2018
(r338316)
@@ -50,6 +50,7 @@ static struct vt_driver vt_fb_driver = {
.vd_fini = vt_fb_fini,
.vd_blank = vt_fb_blank,
.vd_bitblt_text = vt_fb_bitblt_text,
+   .vd_invalidate_text = vt_fb_invalidate_text,
.vd_bitblt_bmp = vt_fb_bitblt_bitmap,
.vd_drawrect = vt_fb_drawrect,
.vd_setpixel = vt_fb_setpixel,
@@ -335,6 +336,7 @@ vt_fb_bitblt_text(struct vt_device *vd, const struct v
term_char_t c;
term_color_t fg, bg;
const uint8_t *pattern;
+   size_t z;
 
vf = vw->vw_font;
 
@@ -351,9 +353,22 @@ vt_fb_bitblt_text(struct vt_device *vd, const struct v
vt_determine_colors(c,
VTBUF_ISCURSOR(>vw_buf, row, col), , );
 
+   z = row * PIXEL_WIDTH(VT_FB_MAX_WIDTH) + col;
+   if (vd->vd_drawn && (vd->vd_drawn[z] == c) &&
+   vd->vd_drawnfg && (vd->vd_drawnfg[z] == fg) &&
+   vd->vd_drawnbg && (vd->vd_drawnbg[z] == bg))
+   continue;
+
vt_fb_bitblt_bitmap(vd, vw,
pattern, NULL, vf->vf_width, vf->vf_height,
x, y, fg, bg);
+
+   if (vd->vd_drawn)
+   vd->vd_drawn[z] = c;
+   if (vd->vd_drawnfg)
+   vd->vd_drawnfg[z] = fg;
+   if (vd->vd_drawnbg)
+   vd->vd_drawnbg[z] = bg;
}
}
 
@@ -377,6 +392,26 @@ vt_fb_bitblt_text(struct vt_device *vd, const struct v
vd->vd_mcursor_fg, vd->vd_mcursor_bg);
}
 #endif
+}
+
+void

svn commit: r338141 - head/release

[Colin Percival]

Author: cperciva
Date: Tue Aug 21 15:30:47 2018
New Revision: 338141
URL: https://svnweb.freebsd.org/changeset/base/338141

Log:
  Quieten the svn (or svnlite) commands used to extract information from an
  SVN checkout for placement into an EC2 AMI.  We only run these if there
  is a .svn directory; but in the event that SVN was used to check out a
  tree which is then exported over NFS, we were unnecessarily noisy.
  
  Reported by:  Andrey Fesenko
  MFC after:3 days
  X-MFC-With:   r336420, r336433, r336593, r336621,
r336622, r336624, r337394, r337401

Modified:
  head/release/Makefile.ec2

Modified: head/release/Makefile.ec2
==
--- head/release/Makefile.ec2   Tue Aug 21 15:11:43 2018(r338140)
+++ head/release/Makefile.ec2   Tue Aug 21 15:30:47 2018(r338141)
@@ -17,11 +17,11 @@ SVN_CMD=   ${_P}/${_S}
 .endif
 .if exists(${SRCTOP}/.svn)
 .  if empty(EC2_SVNBRANCH)
-   EC2_SVNBRANCH!= ${SVN_CMD} info --show-item relative-url ${WORLDDIR} | 
sed -e 's/\^\///'
+   EC2_SVNBRANCH!= ${SVN_CMD} info --show-item relative-url ${WORLDDIR} 
2>/dev/null | sed -e 's/\^\///'
 .  export EC2_SVNBRANCH
 .  endif
 .  if empty(EC2_SVNREV)
-   EC2_SVNREV!=${SVN_CMD} info --show-item last-changed-revision 
${WORLDDIR}
+   EC2_SVNREV!=${SVN_CMD} info --show-item last-changed-revision 
${WORLDDIR} 2>/dev/null || true
 .  export EC2_SVNREV
 .  endif
 .else
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r337411 - head/sys/dev/vt/hw/vga

[Colin Percival]

Author: cperciva
Date: Tue Aug  7 08:33:40 2018
New Revision: 337411
URL: https://svnweb.freebsd.org/changeset/base/337411

Log:
  Replace a pair of 8-bit writes to VGA memory with a single 16-bit write.
  
  The VGA "text mode" buffer has a pair of bytes for each character: One
  byte for the character symbol, and an "attribute" byte encoding the
  foreground and background colours.  When updating the screen, we were
  writing these two bytes separately.
  
  On some virtualized systems, every write results in a glyph being redrawn
  into a (graphical) virtual screen; writing these two bytes separately
  results in twice as much work being done to draw characters, whereas if
  we perform a single 16-bit write instead, the character only needs to be
  redrawn once.
  
  On an EC2 c5.4xlarge instance, this change cuts 1.30s from the kernel boot,
  speeding it up from 8.90s to 7.60s.
  
  MFC after:1 week

Modified:
  head/sys/dev/vt/hw/vga/vt_vga.c

Modified: head/sys/dev/vt/hw/vga/vt_vga.c
==
--- head/sys/dev/vt/hw/vga/vt_vga.c Tue Aug  7 07:12:59 2018
(r337410)
+++ head/sys/dev/vt/hw/vga/vt_vga.c Tue Aug  7 08:33:40 2018
(r337411)
@@ -68,6 +68,8 @@ struct vga_softc {
bus_space_read_1(sc->vga_fb_tag, sc->vga_fb_handle, ofs)
 #defineMEM_WRITE1(sc, ofs, val) \
bus_space_write_1(sc->vga_fb_tag, sc->vga_fb_handle, ofs, val)
+#defineMEM_WRITE2(sc, ofs, val) \
+   bus_space_write_2(sc->vga_fb_tag, sc->vga_fb_handle, ofs, val)
 #defineREG_READ1(sc, reg) \
bus_space_read_1(sc->vga_reg_tag, sc->vga_reg_handle, reg)
 #defineREG_WRITE1(sc, reg, val) \
@@ -894,10 +896,8 @@ vga_bitblt_text_txtmode(struct vt_device *vd, const st
cons_to_vga_colors[bg] << 4 |
cons_to_vga_colors[fg];
 
-   MEM_WRITE1(sc, (row * 80 + col) * 2 + 0,
-   ch);
-   MEM_WRITE1(sc, (row * 80 + col) * 2 + 1,
-   attr);
+   MEM_WRITE2(sc, (row * 80 + col) * 2 + 0,
+   ch + ((uint16_t)(attr) << 8));
}
}
 }
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r337401 - head/release

[Colin Percival]

Author: cperciva
Date: Tue Aug  7 00:51:49 2018
New Revision: 337401
URL: https://svnweb.freebsd.org/changeset/base/337401

Log:
  Fix copy-and-paste error in previous commit.

Modified:
  head/release/Makefile.ec2

Modified: head/release/Makefile.ec2
==
--- head/release/Makefile.ec2   Tue Aug  7 00:10:58 2018(r337400)
+++ head/release/Makefile.ec2   Tue Aug  7 00:51:49 2018(r337401)
@@ -36,7 +36,7 @@ AMINAMESUFFIX!=   date +-%Y-%m-%d
 PUBLISH=   --public
 .endif
 .if defined(EC2PUBLICSNAP) && !empty(EC2PUBLICSNAP)
-PUBLISH=   --publicsnap
+PUBLICSNAP=--publicsnap
 .endif
 .if defined(EC2SNSTOPIC) && !empty(EC2SNSTOPIC)
 EC2SNSREL= ${REVISION}-${BRANCH}
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r337394 - head/release

[Colin Percival]

Author: cperciva
Date: Mon Aug  6 19:21:32 2018
New Revision: 337394
URL: https://svnweb.freebsd.org/changeset/base/337394

Log:
  Add EC2PUBLICSNAP option to EC2 builds; this passes a (recently added)
  flag to bsdec2-image-upload instructing it to mark the snapshot of its
  root disk as public (which is independent from marking the created AMIs
  as public).
  
  Requested by: Amazon

Modified:
  head/release/Makefile.ec2

Modified: head/release/Makefile.ec2
==
--- head/release/Makefile.ec2   Mon Aug  6 19:09:55 2018(r337393)
+++ head/release/Makefile.ec2   Mon Aug  6 19:21:32 2018(r337394)
@@ -35,6 +35,9 @@ AMINAMESUFFIX!=   date +-%Y-%m-%d
 .if defined(EC2PUBLIC) && !empty(EC2PUBLIC)
 PUBLISH=   --public
 .endif
+.if defined(EC2PUBLICSNAP) && !empty(EC2PUBLICSNAP)
+PUBLISH=   --publicsnap
+.endif
 .if defined(EC2SNSTOPIC) && !empty(EC2SNSTOPIC)
 EC2SNSREL= ${REVISION}-${BRANCH}
 EC2SNSVERS=${EC2_SVNBRANCH}@${EC2_SVNREV}
@@ -79,7 +82,7 @@ ec2ami: cw-ec2 ${CW_EC2_PORTINSTALL}
@echo "--"
@false
 .endif
-   /usr/local/bin/bsdec2-image-upload ${PUBLISH} --sriov --ena \
+   /usr/local/bin/bsdec2-image-upload ${PUBLISH} ${PUBLICSNAP} --sriov 
--ena \
${.OBJDIR}/ec2.raw \
"${TYPE} ${REVISION}-${BRANCH}-${TARGET}${AMINAMESUFFIX}" \
"${TYPE}/${TARGET} ${EC2_SVNBRANCH}@${EC2_SVNREV}" \
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r336593 - head/release

[Colin Percival]

On 07/22/18 15:53, Matthew Macy wrote:
> That would only fix it if svn weren't installed.

Oops, good point.  Thanks for fixing this, your commits look good to me.

Colin Percival

> On Sun, Jul 22, 2018 at 9:31 AM, Colin Percival  wrote:
>> On 07/22/18 08:04, Rodney W. Grimes wrote:
>>>> Build still works, but you're assuming that developers only use svn.
>>>
>>> No, he correctly assumed that RELEASE engineering only uses svn/svnlite
>>> for building release images.  This Makefile is for production of
>>> official AMIs, official stuff always uses svn.
>>
>> I'm about to head mostly-AFK for the week, but one option here would be
>> to change
>>
>> EC2_SVNBRANCH!=${SVN_CMD} info --show-item relative-url ${WORLDDIR} |
>> sed -e 's/\^\///'
>> EC2_SVNREV!=   ${SVN_CMD} info --show-item last-changed-revision ${WORLDDIR}
>>
>> to
>>
>> .if defined(SVN_CMD)
>> EC2_SVNBRANCH!=${SVN_CMD} info --show-item relative-url ${WORLDDIR} |
>> sed -e 's/\^\///'
>> EC2_SVNREV!=   ${SVN_CMD} info --show-item last-changed-revision ${WORLDDIR}
>> .else
>> EC2_SVNBRANCH=  unknown
>> EC2_SVNREV= unknown
>> .endif
>>
>> I don't know if this will make any difference to anyone using this code,
>> but it should at least silence the warnings for people who aren't checking
>> code out from svn.
>>
>> Colin Percival
>>
>>>> make[1]: "/usr/home/mmacy/devel/freebsd/Makefile.inc1" line 343:
>>>> SYSTEM_COMPILER: libclang will be built for bootstrapping a
>>>> cross-compiler.
>>>> make[1]: "/usr/home/mmacy/devel/freebsd/Makefile.inc1" line 348:
>>>> SYSTEM_LINKER: libclang will be built for bootstrapping a
>>>> cross-linker.
>>>> svn: E155007: '/usr/home/mmacy/devel/freebsd' is not a working copy
>>>> svn: E155007: '/usr/home/mmacy/devel/freebsd' is not a working copy
>>>> make[2]: "/usr/home/mmacy/devel/freebsd/release/Makefile.ec2" line 19:
>>>> warning: "/usr/local/bin/svn info --show-item last-changed-revision
>>>> /usr/home/mmacy/devel/freebsd/release/.." returned non-zero status
>>>> svn: E155007: '/usr/home/mmacy/devel/freebsd' is not a working copy
>>>> svn: E155007: '/usr/home/mmacy/devel/freebsd' is not a working copy
>>>> make[2]: "/usr/home/mmacy/devel/freebsd/release/Makefile.ec2" line 19:
>>>> warning: "/usr/local/bin/svn info --show-item last-changed-revision
>>>> /usr/home/mmacy/devel/freebsd/release/.." returned non-zero status
>>>>
>>>> On Sat, Jul 21, 2018 at 3:54 PM, Colin Percival  
>>>> wrote:
>>>>> Author: cperciva
>>>>> Date: Sat Jul 21 22:54:43 2018
>>>>> New Revision: 336593
>>>>> URL: https://svnweb.freebsd.org/changeset/base/336593
>>>>>
>>>>> Log:
>>>>>   Use svn or svnlite, or ${SVN_CMD} if set, for extracting the SVN branch
>>>>>   and revision number announced in SNS notifications about new EC2 AMIs.
>>>>>
>>>>>   While I'm here, incorporate that information into the AMI "description"
>>>>>   fields, since it's more useful than simply echoing the information
>>>>>   already provided via the AMI "name".
>>>>>
>>>>>   Approved by:  gjb
>>>>>
>>>>> Modified:
>>>>>   head/release/Makefile.ec2
>>>   ^^
>>>
>>>>>
>>>>> Modified: head/release/Makefile.ec2
>>>>> ==
>>>>> --- head/release/Makefile.ec2   Sat Jul 21 22:15:48 2018(r336592)
>>>>> +++ head/release/Makefile.ec2   Sat Jul 21 22:54:43 2018(r336593)
>>>>> @@ -5,6 +5,19 @@
>>>>>  # Makefile for creating an EC2 AMI from a disk image.
>>>>>  #
>>>>>
>>>>> +# Figure out where SVN is
>>>>> +.if !defined(SVN_CMD) || empty(SVN_CMD)
>>>>> +. for _P in /usr/bin /usr/local/bin
>>>>> +.  for _S in svn svnlite
>>>>> +.   if exists(${_P}/${_S})
>>>>> +SVN_CMD=   ${_P}/${_S}
>>>>> +.   endif
>>>>> +.  endfor
>>>>> +. endfor
>>>>> +.endif
>>>>> +EC2_SVNBRANCH!=${SVN_CMD} info --show-item relative-url 
>>>>> ${WORLDDIR} | sed -e 's/\^\///'
>

Re: svn commit: r336593 - head/release

[Colin Percival]

On 07/22/18 08:04, Rodney W. Grimes wrote:
>> Build still works, but you're assuming that developers only use svn.
> 
> No, he correctly assumed that RELEASE engineering only uses svn/svnlite
> for building release images.  This Makefile is for production of
> official AMIs, official stuff always uses svn.

I'm about to head mostly-AFK for the week, but one option here would be
to change

EC2_SVNBRANCH!=${SVN_CMD} info --show-item relative-url ${WORLDDIR} |
sed -e 's/\^\///'
EC2_SVNREV!=   ${SVN_CMD} info --show-item last-changed-revision ${WORLDDIR}

to

.if defined(SVN_CMD)
EC2_SVNBRANCH!=${SVN_CMD} info --show-item relative-url ${WORLDDIR} |
sed -e 's/\^\///'
EC2_SVNREV!=   ${SVN_CMD} info --show-item last-changed-revision ${WORLDDIR}
.else
EC2_SVNBRANCH=  unknown
EC2_SVNREV= unknown
.endif

I don't know if this will make any difference to anyone using this code,
but it should at least silence the warnings for people who aren't checking
code out from svn.

Colin Percival

>> make[1]: "/usr/home/mmacy/devel/freebsd/Makefile.inc1" line 343:
>> SYSTEM_COMPILER: libclang will be built for bootstrapping a
>> cross-compiler.
>> make[1]: "/usr/home/mmacy/devel/freebsd/Makefile.inc1" line 348:
>> SYSTEM_LINKER: libclang will be built for bootstrapping a
>> cross-linker.
>> svn: E155007: '/usr/home/mmacy/devel/freebsd' is not a working copy
>> svn: E155007: '/usr/home/mmacy/devel/freebsd' is not a working copy
>> make[2]: "/usr/home/mmacy/devel/freebsd/release/Makefile.ec2" line 19:
>> warning: "/usr/local/bin/svn info --show-item last-changed-revision
>> /usr/home/mmacy/devel/freebsd/release/.." returned non-zero status
>> svn: E155007: '/usr/home/mmacy/devel/freebsd' is not a working copy
>> svn: E155007: '/usr/home/mmacy/devel/freebsd' is not a working copy
>> make[2]: "/usr/home/mmacy/devel/freebsd/release/Makefile.ec2" line 19:
>> warning: "/usr/local/bin/svn info --show-item last-changed-revision
>> /usr/home/mmacy/devel/freebsd/release/.." returned non-zero status
>>
>> On Sat, Jul 21, 2018 at 3:54 PM, Colin Percival  wrote:
>>> Author: cperciva
>>> Date: Sat Jul 21 22:54:43 2018
>>> New Revision: 336593
>>> URL: https://svnweb.freebsd.org/changeset/base/336593
>>>
>>> Log:
>>>   Use svn or svnlite, or ${SVN_CMD} if set, for extracting the SVN branch
>>>   and revision number announced in SNS notifications about new EC2 AMIs.
>>>
>>>   While I'm here, incorporate that information into the AMI "description"
>>>   fields, since it's more useful than simply echoing the information
>>>   already provided via the AMI "name".
>>>
>>>   Approved by:  gjb
>>>
>>> Modified:
>>>   head/release/Makefile.ec2
>   ^^
> 
>>>
>>> Modified: head/release/Makefile.ec2
>>> ==
>>> --- head/release/Makefile.ec2   Sat Jul 21 22:15:48 2018(r336592)
>>> +++ head/release/Makefile.ec2   Sat Jul 21 22:54:43 2018(r336593)
>>> @@ -5,6 +5,19 @@
>>>  # Makefile for creating an EC2 AMI from a disk image.
>>>  #
>>>
>>> +# Figure out where SVN is
>>> +.if !defined(SVN_CMD) || empty(SVN_CMD)
>>> +. for _P in /usr/bin /usr/local/bin
>>> +.  for _S in svn svnlite
>>> +.   if exists(${_P}/${_S})
>>> +SVN_CMD=   ${_P}/${_S}
>>> +.   endif
>>> +.  endfor
>>> +. endfor
>>> +.endif
>>> +EC2_SVNBRANCH!=${SVN_CMD} info --show-item relative-url 
>>> ${WORLDDIR} | sed -e 's/\^\///'
>>> +EC2_SVNREV!=   ${SVN_CMD} info --show-item last-changed-revision 
>>> ${WORLDDIR}
>>> +
>>>  .if ${BRANCH} == "CURRENT" || ${BRANCH} == "STABLE" || ${BRANCH} == 
>>> "PRERELEASE"
>>>  AMINAMESUFFIX!=date +-%Y-%m-%d
>>>  .endif
>>> @@ -13,8 +26,6 @@ PUBLISH=  --public
>>>  .endif
>>>  .if defined(EC2SNSTOPIC) && !empty(EC2SNSTOPIC)
>>>  EC2SNSREL= ${REVISION}-${BRANCH}
>>> -EC2_SVNBRANCH!=svn info --show-item relative-url ${WORLDDIR} | sed 
>>> -e 's/\^\///'
>>> -EC2_SVNREV!=   svn info --show-item last-changed-revision ${WORLDDIR}
>>>  EC2SNSVERS=${EC2_SVNBRANCH}@${EC2_SVNREV}
>>>  .endif
>>>
>>> @@ -60,7 +71,7 @@ ec2ami: cw-ec2 ${CW_EC2_PORTINSTALL}
>>> /usr/local/bin/bsdec2-image-upload ${PUBLISH} --sriov --ena \
>>> ${.OBJDIR}/ec2.raw \
>>> "${TYPE} ${REVISION}-${BRANCH}-${TARGET}${AMINAMESUFFIX}" \
>>> -   "${TYPE} ${REVISION}-${BRANCH}-${TARGET}" \
>>> +   "${TYPE}/${TARGET} ${EC2_SVNBRANCH}@${EC2_SVNREV}" \
>>> ${AWSREGION} ${AWSBUCKET} ${AWSKEYFILE} \
>>> ${EC2SNSTOPIC} ${EC2SNSREL} ${EC2SNSVERS}
>>> @touch ${.TARGET}
>>>
>>
>>
> 

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r336593 - head/release

[Colin Percival]

Author: cperciva
Date: Sat Jul 21 22:54:43 2018
New Revision: 336593
URL: https://svnweb.freebsd.org/changeset/base/336593

Log:
  Use svn or svnlite, or ${SVN_CMD} if set, for extracting the SVN branch
  and revision number announced in SNS notifications about new EC2 AMIs.
  
  While I'm here, incorporate that information into the AMI "description"
  fields, since it's more useful than simply echoing the information
  already provided via the AMI "name".
  
  Approved by:  gjb

Modified:
  head/release/Makefile.ec2

Modified: head/release/Makefile.ec2
==
--- head/release/Makefile.ec2   Sat Jul 21 22:15:48 2018(r336592)
+++ head/release/Makefile.ec2   Sat Jul 21 22:54:43 2018(r336593)
@@ -5,6 +5,19 @@
 # Makefile for creating an EC2 AMI from a disk image.
 #
 
+# Figure out where SVN is
+.if !defined(SVN_CMD) || empty(SVN_CMD)
+. for _P in /usr/bin /usr/local/bin
+.  for _S in svn svnlite
+.   if exists(${_P}/${_S})
+SVN_CMD=   ${_P}/${_S}
+.   endif
+.  endfor
+. endfor
+.endif
+EC2_SVNBRANCH!=${SVN_CMD} info --show-item relative-url ${WORLDDIR} | 
sed -e 's/\^\///'
+EC2_SVNREV!=   ${SVN_CMD} info --show-item last-changed-revision ${WORLDDIR}
+
 .if ${BRANCH} == "CURRENT" || ${BRANCH} == "STABLE" || ${BRANCH} == 
"PRERELEASE"
 AMINAMESUFFIX!=date +-%Y-%m-%d
 .endif
@@ -13,8 +26,6 @@ PUBLISH=  --public
 .endif
 .if defined(EC2SNSTOPIC) && !empty(EC2SNSTOPIC)
 EC2SNSREL= ${REVISION}-${BRANCH}
-EC2_SVNBRANCH!=svn info --show-item relative-url ${WORLDDIR} | sed -e 
's/\^\///'
-EC2_SVNREV!=   svn info --show-item last-changed-revision ${WORLDDIR}
 EC2SNSVERS=${EC2_SVNBRANCH}@${EC2_SVNREV}
 .endif
 
@@ -60,7 +71,7 @@ ec2ami: cw-ec2 ${CW_EC2_PORTINSTALL}
/usr/local/bin/bsdec2-image-upload ${PUBLISH} --sriov --ena \
${.OBJDIR}/ec2.raw \
"${TYPE} ${REVISION}-${BRANCH}-${TARGET}${AMINAMESUFFIX}" \
-   "${TYPE} ${REVISION}-${BRANCH}-${TARGET}" \
+   "${TYPE}/${TARGET} ${EC2_SVNBRANCH}@${EC2_SVNREV}" \
${AWSREGION} ${AWSBUCKET} ${AWSKEYFILE} \
${EC2SNSTOPIC} ${EC2SNSREL} ${EC2SNSVERS}
@touch ${.TARGET}
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r336433 - head/release

[Colin Percival]

Author: cperciva
Date: Tue Jul 17 23:16:05 2018
New Revision: 336433
URL: https://svnweb.freebsd.org/changeset/base/336433

Log:
  Check that EC2SNSTOPIC and EC2PUBLIC are non-empty, along with being
  defined, in order to avoid problems with thermite.sh.
  
  Reported by:  gjb

Modified:
  head/release/Makefile.ec2

Modified: head/release/Makefile.ec2
==
--- head/release/Makefile.ec2   Tue Jul 17 23:00:52 2018(r336432)
+++ head/release/Makefile.ec2   Tue Jul 17 23:16:05 2018(r336433)
@@ -8,10 +8,10 @@
 .if ${BRANCH} == "CURRENT" || ${BRANCH} == "STABLE" || ${BRANCH} == 
"PRERELEASE"
 AMINAMESUFFIX!=date +-%Y-%m-%d
 .endif
-.if defined(EC2PUBLIC)
+.if defined(EC2PUBLIC) && !empty(EC2PUBLIC)
 PUBLISH=   --public
 .endif
-.if defined(EC2SNSTOPIC)
+.if defined(EC2SNSTOPIC) && !empty(EC2SNSTOPIC)
 EC2SNSREL= ${REVISION}-${BRANCH}
 EC2_SVNBRANCH!=svn info --show-item relative-url ${WORLDDIR} | sed -e 
's/\^\///'
 EC2_SVNREV!=   svn info --show-item last-changed-revision ${WORLDDIR}
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r336420 - head/release

[Colin Percival]

Author: cperciva
Date: Tue Jul 17 19:05:30 2018
New Revision: 336420
URL: https://svnweb.freebsd.org/changeset/base/336420

Log:
  Teach EC2 AMI release code to send an SNS notification after publishing
  new AMIs if EC2SNSTOPIC is defined.
  
  Reviewed by:  gjb
  Requested by: Amazon

Modified:
  head/release/Makefile.ec2

Modified: head/release/Makefile.ec2
==
--- head/release/Makefile.ec2   Tue Jul 17 15:16:34 2018(r336419)
+++ head/release/Makefile.ec2   Tue Jul 17 19:05:30 2018(r336420)
@@ -11,6 +11,12 @@ AMINAMESUFFIX!=  date +-%Y-%m-%d
 .if defined(EC2PUBLIC)
 PUBLISH=   --public
 .endif
+.if defined(EC2SNSTOPIC)
+EC2SNSREL= ${REVISION}-${BRANCH}
+EC2_SVNBRANCH!=svn info --show-item relative-url ${WORLDDIR} | sed -e 
's/\^\///'
+EC2_SVNREV!=   svn info --show-item last-changed-revision ${WORLDDIR}
+EC2SNSVERS=${EC2_SVNBRANCH}@${EC2_SVNREV}
+.endif
 
 CLEANFILES+=   ec2ami
 
@@ -55,5 +61,6 @@ ec2ami: cw-ec2 ${CW_EC2_PORTINSTALL}
${.OBJDIR}/ec2.raw \
"${TYPE} ${REVISION}-${BRANCH}-${TARGET}${AMINAMESUFFIX}" \
"${TYPE} ${REVISION}-${BRANCH}-${TARGET}" \
-   ${AWSREGION} ${AWSBUCKET} ${AWSKEYFILE}
+   ${AWSREGION} ${AWSBUCKET} ${AWSKEYFILE} \
+   ${EC2SNSTOPIC} ${EC2SNSREL} ${EC2SNSVERS}
@touch ${.TARGET}
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r335553 - head/sys/kern

[Colin Percival]

Author: cperciva
Date: Fri Jun 22 10:23:32 2018
New Revision: 335553
URL: https://svnweb.freebsd.org/changeset/base/335553

Log:
  Improve the accuracy of the POSIX "process CPU-time" clocks by adding the
  used portion of the current thread's time slice if the current thread
  belongs to the process being queried (i.e., if clock_gettime is invoked
  with a clock ID of CLOCK_PROCESS_CPUTIME_ID or the value provided by
  passing getpid(2) to clock_getcpuclockid(3)).
  
  The CLOCK_VIRTUAL and CLOCK_PROF timers already make this adjustment via
  long-standing code in calcru(), but since those timers are not specified
  by POSIX it seems useful to add it here so that the higher accuracy is
  available to code which aims to be portable.
  
  PR:   228669
  Reported by:  Graham Percival
  Reviewed by:  kib
  MFC after:1 week

Modified:
  head/sys/kern/kern_time.c

Modified: head/sys/kern/kern_time.c
==
--- head/sys/kern/kern_time.c   Fri Jun 22 10:20:21 2018(r335552)
+++ head/sys/kern/kern_time.c   Fri Jun 22 10:23:32 2018(r335553)
@@ -280,6 +280,8 @@ get_process_cputime(struct proc *targetp, struct times
PROC_STATLOCK(targetp);
rufetch(targetp, );
runtime = targetp->p_rux.rux_runtime;
+   if (curthread->td_proc == targetp)
+   runtime += cpu_ticks() - PCPU_GET(switchtime);
PROC_STATUNLOCK(targetp);
cputick2timespec(runtime, ats);
 }
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r332663 - head/release/tools

[Colin Percival]

Author: cperciva
Date: Wed Apr 18 05:58:27 2018
New Revision: 332663
URL: https://svnweb.freebsd.org/changeset/base/332663

Log:
  Move debug.{trace,debugger}_on_panic and kern.panic_reboot_wait_time in
  EC2 instances from sysctl.conf to loader.conf; these can all be set as
  loader tunables, and setting them in loader.conf gives us the right
  behaviour in the event of a kernel panic taking place prior to when
  sysctl.conf is processed.
  
  MFC after:1 week

Modified:
  head/release/tools/ec2.conf

Modified: head/release/tools/ec2.conf
==
--- head/release/tools/ec2.conf Wed Apr 18 02:25:03 2018(r332662)
+++ head/release/tools/ec2.conf Wed Apr 18 05:58:27 2018(r332663)
@@ -60,9 +60,9 @@ vm_extra_pre_umount() {
# The EC2 console is output-only, so while printing a backtrace can
# be useful, there's no point dropping into a debugger or waiting
# for a keypress.
-   echo 'debug.trace_on_panic=1' >> ${DESTDIR}/etc/sysctl.conf
-   echo 'debug.debugger_on_panic=0' >> ${DESTDIR}/etc/sysctl.conf
-   echo 'kern.panic_reboot_wait_time=0' >> ${DESTDIR}/etc/sysctl.conf
+   echo 'debug.trace_on_panic=1' >> ${DESTDIR}/boot/loader.conf
+   echo 'debug.debugger_on_panic=0' >> ${DESTDIR}/boot/loader.conf
+   echo 'kern.panic_reboot_wait_time=0' >> ${DESTDIR}/boot/loader.conf
 
# The console is not interactive, so we might as well boot quickly.
echo 'autoboot_delay="-1"' >> ${DESTDIR}/boot/loader.conf
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r330236 - head/sys/compat/linuxkpi/common/src

[Colin Percival]

On 03/01/18 02:31, Hans Petter Selasky wrote:
> New Revision: 330236
>   Correct the return value from flush_work() and flush_delayed_work() in the
>   LinuxKPI to comply more with Linux. This fixes an issue when these functions
>   are used in waiting loops.
>   
>   MFC after:  1 week
>   Sponsored by:   Mellanox Technologies

In case the data point is useful: Merging this to stable/11 unbroke S3
suspend/resume on my laptop (system76 Galago Pro, kabylake, using the
drm-next-kmod port).

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r329737 - head/stand/i386/boot2

[Colin Percival]

On 02/21/18 10:10, Benno Rice wrote:
>   Curiously, changing whitespace seems to cause the md5 of the .o files to 
> differ
>   these days hence the following testing strategy:
>   
>   Tested by:  objdump -d | md5 (both in-tree clang and lang/gcc6)

Is this simply because line numbers are changing?  That isn't new; I remember
a case where a security advisory touched a .h file and suddenly a huge number
of binaries changed because they included header file line numbers.

>  static inline int
>  strcmp(const char *s1, const char *s2)
>  {
> -for (; *s1 == *s2 && *s1; s1++, s2++);
> -return (unsigned char)*s1 - (unsigned char)*s2;
> +
> + for (; *s1 == *s2 && *s1; s1++, s2++);
> + return (unsigned char)*s1 - (unsigned char)*s2;
>  }

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r327423 - in head/sys: kern sys

[Colin Percival]

On 01/17/18 15:09, Gleb Smirnoff wrote:
> On Sun, Dec 31, 2017 at 09:21:01AM +0000, Colin Percival wrote:
> C>   Code for recording timestamps of events, especially function 
> entries/exits.
> C>   This is a very primitive system, intended for use in measuring 
> performance
> C>   during the early system boot, before more sophisticated tools like DTrace
> C>   or infrastructure like kernel memory allocation and mutexes are 
> available.
> 
> Sorry if my question is too lame and late. Why can't you use ktr(4) tracer
> for that purpose? The discussion on freebsd-current also doesn't have answer.

I considered it, but it wasn't a good fit for a couple reasons:

1. ktr uses a circular buffer, while (since I'm looking at the boot process) I
want to fill a buffer and then stop recording.

2. I would have needed to hack up ktr in order to make it work early enough
in the boot process.  (Most obviously: When we enter hammer_time, trying to
access 'curthread' is a Bad Idea.)

Since the timestamp recording framework I needed was just a few lines of code
-- all the other commits are to add the points where timestamps are recorded,
and would have been needed even if I used ktr -- it seemed better to just add
my own code rather than coercing ktr to do what I needed.

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r327447 - head/sys/sys

[Colin Percival]

Author: cperciva
Date: Sun Dec 31 21:00:21 2017
New Revision: 327447
URL: https://svnweb.freebsd.org/changeset/base/327447

Log:
  Wrap includes in sys/tslog.h with #ifdef TSLOG.
  
  This is necessary because some non-kernel code #defines _KERNEL and then
  includes kernel headers; as a result, it was getting conflicting versions
  of curthread and curproc.  Non-kernel code should probably refrain from
  defining _KERNEL, but for now hiding these indirect inclusions fixes the
  build.
  
  Reported by:  Michael Butler, Herbert J. Skuhra

Modified:
  head/sys/sys/tslog.h

Modified: head/sys/sys/tslog.h
==
--- head/sys/sys/tslog.hSun Dec 31 20:30:51 2017(r327446)
+++ head/sys/sys/tslog.hSun Dec 31 21:00:21 2017(r327447)
@@ -29,8 +29,10 @@
 #ifndef _TSLOG_H_
 #define_TSLOG_H_
 
+#ifdef TSLOG
 #include 
 #include 
+#endif
 
 #define TS_ENTER   0
 #define TS_EXIT1
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r327432 - in head/sys: arm/amlogic/aml8726 arm/arm arm/at91 arm/cavium/cns11xx arm/lpc arm/mv arm/ralink arm/xscale/i8134x arm/xscale/ixp425 arm/xscale/pxa kern mips/ingenic mips/mips m...

[Colin Percival]

Author: cperciva
Date: Sun Dec 31 09:24:41 2017
New Revision: 327432
URL: https://svnweb.freebsd.org/changeset/base/327432

Log:
  Use the TSLOG framework to record entry/exit timestamps for DELAY and
  _vprintf; these functions are called in many places and can contribute
  meaningfully to the total time spent booting.

Modified:
  head/sys/arm/amlogic/aml8726/aml8726_timer.c
  head/sys/arm/arm/generic_timer.c
  head/sys/arm/arm/machdep.c
  head/sys/arm/arm/mpcore_timer.c
  head/sys/arm/at91/at91_machdep.c
  head/sys/arm/cavium/cns11xx/timer.c
  head/sys/arm/lpc/lpc_timer.c
  head/sys/arm/mv/timer.c
  head/sys/arm/ralink/rt1310_timer.c
  head/sys/arm/xscale/i8134x/i80321_timer.c
  head/sys/arm/xscale/ixp425/ixp425_timer.c
  head/sys/arm/xscale/pxa/pxa_timer.c
  head/sys/kern/subr_prf.c
  head/sys/mips/ingenic/jz4780_timer.c
  head/sys/mips/mips/tick.c
  head/sys/mips/nlm/tick.c
  head/sys/mips/rmi/tick.c
  head/sys/powerpc/powerpc/clock.c
  head/sys/riscv/riscv/timer.c
  head/sys/sparc64/sparc64/clock.c
  head/sys/x86/x86/delay.c

Modified: head/sys/arm/amlogic/aml8726/aml8726_timer.c
==
--- head/sys/arm/amlogic/aml8726/aml8726_timer.cSun Dec 31 09:24:11 
2017(r327431)
+++ head/sys/arm/amlogic/aml8726/aml8726_timer.cSun Dec 31 09:24:41 
2017(r327432)
@@ -347,6 +347,7 @@ DELAY(int usec)
}
return;
}
+   TSENTER();
 
/*
 * Some of the other timers in the source tree do this calculation as:
@@ -391,4 +392,5 @@ DELAY(int usec)
previous = now;
remaining -= delta;
}
+   TSEXIT();
 }

Modified: head/sys/arm/arm/generic_timer.c
==
--- head/sys/arm/arm/generic_timer.cSun Dec 31 09:24:11 2017
(r327431)
+++ head/sys/arm/arm/generic_timer.cSun Dec 31 09:24:41 2017
(r327432)
@@ -526,6 +526,7 @@ DELAY(int usec)
 {
int32_t counts;
 
+   TSENTER();
/*
 * Check the timers are setup, if not just
 * use a for loop for the meantime
@@ -540,6 +541,7 @@ DELAY(int usec)
cpufunc_nullop();
} else
arm_tmr_do_delay(usec, arm_tmr_sc);
+   TSEXIT();
 }
 #endif
 

Modified: head/sys/arm/arm/machdep.c
==
--- head/sys/arm/arm/machdep.c  Sun Dec 31 09:24:11 2017(r327431)
+++ head/sys/arm/arm/machdep.c  Sun Dec 31 09:24:41 2017(r327432)
@@ -347,7 +347,9 @@ void
 DELAY(int usec)
 {
 
+   TSENTER();
delay_impl(usec, delay_arg);
+   TSEXIT();
 }
 #endif
 

Modified: head/sys/arm/arm/mpcore_timer.c
==
--- head/sys/arm/arm/mpcore_timer.c Sun Dec 31 09:24:11 2017
(r327431)
+++ head/sys/arm/arm/mpcore_timer.c Sun Dec 31 09:24:41 2017
(r327432)
@@ -547,6 +547,7 @@ DELAY(int usec)
struct arm_tmr_softc *sc;
int32_t counts;
 
+   TSENTER();
/* Check the timers are setup, if not just use a for loop for the 
meantime */
if (arm_tmr_tc == NULL || arm_tmr_timecount.tc_frequency == 0) {
for (; usec > 0; usec--)
@@ -558,5 +559,6 @@ DELAY(int usec)
sc = arm_tmr_tc->tc_priv;
arm_tmr_delay(usec, sc);
}
+   TSEXIT();
 }
 #endif

Modified: head/sys/arm/at91/at91_machdep.c
==
--- head/sys/arm/at91/at91_machdep.cSun Dec 31 09:24:11 2017
(r327431)
+++ head/sys/arm/at91/at91_machdep.cSun Dec 31 09:24:41 2017
(r327432)
@@ -672,8 +672,10 @@ void
 DELAY(int n)
 {
 
+   TSENTER();
if (soc_info.soc_data)
soc_info.soc_data->soc_delay(n);
+   TSEXIT();
 }
 
 void

Modified: head/sys/arm/cavium/cns11xx/timer.c
==
--- head/sys/arm/cavium/cns11xx/timer.c Sun Dec 31 09:24:11 2017
(r327431)
+++ head/sys/arm/cavium/cns11xx/timer.c Sun Dec 31 09:24:41 2017
(r327432)
@@ -122,6 +122,7 @@ DELAY(int usec)
;
return;
}
+   TSENTER();
 
val = read_timer_counter_noint();
nticks = (((APB_clock / 1000) * usec) / 1000) + 100;
@@ -135,7 +136,7 @@ DELAY(int usec)
 
val = val_temp;
}
-
+   TSEXIT();
 }
 
 /*

Modified: head/sys/arm/lpc/lpc_timer.c
==
--- head/sys/arm/lpc/lpc_timer.cSun Dec 31 09:24:11 2017
(r327431)
+++ head/sys/arm/lpc/lpc_timer.cSun Dec 31 09:24:41 2017
(r327432)
@@ -292,6 +292,7 @@ DELAY(int usec)
;

svn commit: r327431 - head/sys/kern

[Colin Percival]

Author: cperciva
Date: Sun Dec 31 09:24:11 2017
New Revision: 327431
URL: https://svnweb.freebsd.org/changeset/base/327431

Log:
  Instrument thread creations for the the benefit of the TSLOG framework.
  This assists in tracking time spent while the boot is being "held" waiting
  for something to happen.

Modified:
  head/sys/kern/kern_kthread.c

Modified: head/sys/kern/kern_kthread.c
==
--- head/sys/kern/kern_kthread.cSun Dec 31 09:23:52 2017
(r327430)
+++ head/sys/kern/kern_kthread.cSun Dec 31 09:24:11 2017
(r327431)
@@ -44,6 +44,7 @@ __FBSDID("$FreeBSD$");
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 
@@ -124,6 +125,7 @@ kproc_create(void (*func)(void *), void *arg,
 #ifdef KTR
sched_clear_tdname(td);
 #endif
+   TSTHREAD(td, td->td_name);
 
/* call the processes' main()... */
cpu_fork_kthread_handler(td, func, arg);
@@ -282,6 +284,8 @@ kthread_add(void (*func)(void *), void *arg, struct pr
va_start(ap, fmt);
vsnprintf(newtd->td_name, sizeof(newtd->td_name), fmt, ap);
va_end(ap);
+
+   TSTHREAD(newtd, newtd->td_name);
 
newtd->td_proc = p;  /* needed for cpu_copy_thread */
/* might be further optimized for kthread */
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r327430 - in head/sys: geom kern

[Colin Percival]

Author: cperciva
Date: Sun Dec 31 09:23:52 2017
New Revision: 327430
URL: https://svnweb.freebsd.org/changeset/base/327430

Log:
  Instrument "boot holds" for the benefit of the TSLOG framework.  These
  are places where the "main thread" of the booting kernel (either the
  thread which later becomes swapper or the thread which later becomes
  init) has to stop and wait for action to take place in another thread
  before continuing.
  
  There are currently three such holds:
  1. The intr_config_hooks SYSINIT waits for hooks registered via the
  config_intrhook_establish function; this allows (typically) devices
  which need interrupts enabled to complete their initialization to do
  so before root is mounted.
  
  2. The g_waitidle function waits for the GEOM event queue to be empty;
  this ensures that all of the disks which have been attached have been
  tasted before we attempt to mount root.
  
  3. The vfs_mountroot_wait function (in addition to calling g_waitidle)
  waits for holds registered via root_mount_hold; among other things, this
  is used by the USB subsystem to ensure that we don't fail to mount root
  if it's located on a USB disk which takes a while to probe.

Modified:
  head/sys/geom/geom_event.c
  head/sys/kern/subr_autoconf.c
  head/sys/kern/vfs_mountroot.c

Modified: head/sys/geom/geom_event.c
==
--- head/sys/geom/geom_event.c  Sun Dec 31 09:23:35 2017(r327429)
+++ head/sys/geom/geom_event.c  Sun Dec 31 09:23:52 2017(r327430)
@@ -87,9 +87,11 @@ g_waitidle(void)
g_topology_assert_not();
 
mtx_lock(_eventlock);
+   TSWAIT("GEOM events");
while (!TAILQ_EMPTY(_events))
msleep(_pending_events, _eventlock, PPAUSE,
"g_waitidle", hz/5);
+   TSUNWAIT("GEOM events");
mtx_unlock(_eventlock);
curthread->td_pflags &= ~TDP_GEOM;
 }
@@ -266,6 +268,7 @@ one_event(void)
ep->func(ep->arg, 0);
g_topology_assert();
mtx_lock(_eventlock);
+   TSRELEASE("GEOM events");
TAILQ_REMOVE(_events, ep, events);
ep->flag &= ~EV_INPROGRESS;
if (ep->flag & EV_WAKEUP) {
@@ -324,6 +327,7 @@ g_cancel_event(void *ref)
break;
if (ep->ref[n] != ref)
continue;
+   TSRELEASE("GEOM events");
TAILQ_REMOVE(_events, ep, events);
ep->func(ep->arg, EV_CANCEL);
mtx_assert(_eventlock, MA_OWNED);
@@ -367,6 +371,7 @@ g_post_event_x(g_event_t *func, void *arg, int flag, i
ep->func = func;
ep->arg = arg;
mtx_lock(_eventlock);
+   TSHOLD("GEOM events");
TAILQ_INSERT_TAIL(_events, ep, events);
mtx_unlock(_eventlock);
wakeup(_wait_event);

Modified: head/sys/kern/subr_autoconf.c
==
--- head/sys/kern/subr_autoconf.c   Sun Dec 31 09:23:35 2017
(r327429)
+++ head/sys/kern/subr_autoconf.c   Sun Dec 31 09:23:52 2017
(r327430)
@@ -155,6 +155,7 @@ boot_run_interrupt_driven_config_hooks(void *dummy)
run_interrupt_driven_config_hooks();
 
/* Block boot processing until all hooks are disestablished. */
+   TSWAIT("config hooks");
mtx_lock(_config_hook_lock);
warned = 0;
while (!TAILQ_EMPTY(_config_hook_list)) {
@@ -168,6 +169,7 @@ boot_run_interrupt_driven_config_hooks(void *dummy)
}
}
mtx_unlock(_config_hook_lock);
+   TSUNWAIT("config hooks");
 }
 
 SYSINIT(intr_config_hooks, SI_SUB_INT_CONFIG_HOOKS, SI_ORDER_FIRST,
@@ -183,6 +185,7 @@ config_intrhook_establish(struct intr_config_hook *hoo
 {
struct intr_config_hook *hook_entry;
 
+   TSHOLD("config hooks");
mtx_lock(_config_hook_lock);
TAILQ_FOREACH(hook_entry, _config_hook_list, ich_links)
if (hook_entry == hook)
@@ -239,6 +242,7 @@ config_intrhook_disestablish(struct intr_config_hook *
if (next_to_notify == hook)
next_to_notify = TAILQ_NEXT(hook, ich_links);
TAILQ_REMOVE(_config_hook_list, hook, ich_links);
+   TSRELEASE("config hooks");
 
/* Wakeup anyone watching the list */
wakeup(_config_hook_list);

Modified: head/sys/kern/vfs_mountroot.c
==
--- head/sys/kern/vfs_mountroot.c   Sun Dec 31 09:23:35 2017
(r327429)
+++ head/sys/kern/vfs_mountroot.c   Sun Dec 31 09:23:52 2017
(r327430)
@@ -176,6 +176,7 @@ root_mount_hold(const char *identifier)
h = malloc(sizeof *h, M_DEVBUF, M_ZERO | M_WAITOK);
h->who = identifier;
mtx_lock(_holds_mtx);
+   TSHOLD("root mount");
LIST_INSERT_HEAD(_holds, h, list);
mtx_unlock(_holds_mtx);

svn commit: r327429 - head/sys/sys

[Colin Percival]

Author: cperciva
Date: Sun Dec 31 09:23:35 2017
New Revision: 327429
URL: https://svnweb.freebsd.org/changeset/base/327429

Log:
  Use the TSLOG framework to record entry/exit timestamps for VFS_MOUNT calls.

Modified:
  head/sys/sys/mount.h

Modified: head/sys/sys/mount.h
==
--- head/sys/sys/mount.hSun Dec 31 09:23:19 2017(r327428)
+++ head/sys/sys/mount.hSun Dec 31 09:23:35 2017(r327429)
@@ -40,6 +40,7 @@
 #ifdef _KERNEL
 #include 
 #include 
+#include 
 #include 
 #include 
 #endif
@@ -708,9 +709,11 @@ vfs_statfs_t   __vfs_statfs;
 #defineVFS_MOUNT(MP) ({
\
int _rc;\
\
+   TSRAW(curthread, TS_ENTER, "VFS_MOUNT", (MP)->mnt_vfc->vfc_name);\
VFS_PROLOGUE(MP);   \
_rc = (*(MP)->mnt_op->vfs_mount)(MP);   \
VFS_EPILOGUE(MP);   \
+   TSRAW(curthread, TS_EXIT, "VFS_MOUNT", (MP)->mnt_vfc->vfc_name);\
_rc; })
 
 #defineVFS_UNMOUNT(MP, FORCE) ({   
\
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r327428 - in head/sys: kern tools

[Colin Percival]

Author: cperciva
Date: Sun Dec 31 09:23:19 2017
New Revision: 327428
URL: https://svnweb.freebsd.org/changeset/base/327428

Log:
  Teach makeobjops.awk to accept PROLOG and EPILOG blocks before
  METHOD and STATICMETHOD declarations; that code will be inserted
  into the dispatch function before and after the method call.
  
  Use this functionality and the TSLOG framework to record DEVICE_ATTACH
  and DEVICE_PROBE entry/exit timestamps.

Modified:
  head/sys/kern/device_if.m
  head/sys/tools/makeobjops.awk

Modified: head/sys/kern/device_if.m
==
--- head/sys/kern/device_if.m   Sun Dec 31 09:23:02 2017(r327427)
+++ head/sys/kern/device_if.m   Sun Dec 31 09:23:19 2017(r327428)
@@ -39,6 +39,11 @@
  */
 INTERFACE device;
 
+# Needed for timestamping device probe/attach calls
+HEADER {
+   #include 
+}
+
 #
 # Default implementations of some methods.
 #
@@ -142,6 +147,12 @@ CODE {
  * be returned to indicate the type of error
  * @see DEVICE_ATTACH(), pci_get_vendor(), pci_get_device()
  */
+PROLOG {
+   TSENTER2(device_get_name(dev));
+}
+EPILOG {
+   TSEXIT2(device_get_name(dev));
+}
 METHOD int probe {
device_t dev;
 };
@@ -199,6 +210,12 @@ STATICMETHOD void identify {
  * be returned to indicate the type of error
  * @see DEVICE_PROBE()
  */
+PROLOG {
+   TSENTER2(device_get_name(dev));
+}
+EPILOG {
+   TSEXIT2(device_get_name(dev));
+}
 METHOD int attach {
device_t dev;
 };

Modified: head/sys/tools/makeobjops.awk
==
--- head/sys/tools/makeobjops.awk   Sun Dec 31 09:23:02 2017
(r327427)
+++ head/sys/tools/makeobjops.awk   Sun Dec 31 09:23:19 2017
(r327428)
@@ -326,11 +326,19 @@ function handle_method (static, doc)
}
printh("{");
printh("\tkobjop_t _m;");
+   if (ret != "void")
+   printh("\t" ret " rc;");
if (!static)
firstvar = "((kobj_t)" firstvar ")";
+   if (prolog != "")
+   printh(prolog);
printh("\tKOBJOPLOOKUP(" firstvar "->ops," mname ");");
-   retrn =  (ret != "void") ? "return " : "";
-   printh("\t" retrn "((" mname "_t *) _m)(" varname_list ");");
+   rceq = (ret != "void") ? "rc = " : "";
+   printh("\t" rceq "((" mname "_t *) _m)(" varname_list ");");
+   if (epilog != "")
+   printh(epilog);
+   if (ret != "void")
+   printh("\treturn (rc);");
printh("}\n");
 }
 
@@ -440,6 +448,8 @@ for (file_i = 0; file_i < num_files; file_i++) {
lineno = 0;
error = 0;  # to signal clean up and gerror setting
lastdoc = "";
+   prolog = "";
+   epilog = "";
 
while (!error && (getline < src) > 0) {
lineno++;
@@ -473,10 +483,18 @@ for (file_i = 0; file_i < num_files; file_i++) {
else if (/^METHOD/) {
handle_method(0, lastdoc);
lastdoc = "";
+   prolog = "";
+   epilog = "";
} else if (/^STATICMETHOD/) {
handle_method(1, lastdoc);
lastdoc = "";
-   } else {
+   prolog = "";
+   epilog = "";
+   } else if (/^PROLOG[]*{$/)
+   prolog = handle_code();
+   else if (/^EPILOG[  ]*{$/)
+   epilog = handle_code();
+   else {
debug($0);
warnsrc("Invalid line encountered");
error = 1;
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r327427 - head/sys/sys

[Colin Percival]

Author: cperciva
Date: Sun Dec 31 09:23:02 2017
New Revision: 327427
URL: https://svnweb.freebsd.org/changeset/base/327427

Log:
  Use the TSLOG framework to record SYSINIT entry/exit timestamps.

Modified:
  head/sys/sys/kernel.h

Modified: head/sys/sys/kernel.h
==
--- head/sys/sys/kernel.h   Sun Dec 31 09:22:31 2017(r327426)
+++ head/sys/sys/kernel.h   Sun Dec 31 09:23:02 2017(r327427)
@@ -54,6 +54,9 @@
 /* for intrhook below */
 #include 
 
+/* for timestamping SYSINITs; other files may assume this is included here */
+#include 
+
 /* Global variables for the kernel. */
 
 /* 1.1 */
@@ -229,14 +232,44 @@ struct sysinit {
  * correct warnings when -Wcast-qual is used.
  *
  */
+#ifdef TSLOG
+struct sysinit_tslog {
+   sysinit_cfunc_t func;
+   const void * data;
+   const char * name;
+};
+static inline void
+sysinit_tslog_shim(const void * data)
+{
+   const struct sysinit_tslog * x = data;
+
+   TSRAW(curthread, TS_ENTER, "SYSINIT", x->name);
+   (x->func)(x->data);
+   TSRAW(curthread, TS_EXIT, "SYSINIT", x->name);
+}
 #defineC_SYSINIT(uniquifier, subsystem, order, func, ident)\
+   static struct sysinit_tslog uniquifier ## _sys_init_tslog = {   \
+   func,   \
+   (ident),\
+   #uniquifier \
+   };  \
static struct sysinit uniquifier ## _sys_init = {   \
subsystem,  \
order,  \
+   sysinit_tslog_shim, \
+## _sys_init_tslog  \
+   };  \
+   DATA_SET(sysinit_set,uniquifier ## _sys_init)
+#else
+#defineC_SYSINIT(uniquifier, subsystem, order, func, ident)\
+   static struct sysinit uniquifier ## _sys_init = {   \
+   subsystem,  \
+   order,  \
func,   \
(ident) \
};  \
DATA_SET(sysinit_set,uniquifier ## _sys_init)
+#endif
 
 #defineSYSINIT(uniquifier, subsystem, order, func, ident)  \
C_SYSINIT(uniquifier, subsystem, order, \
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r327426 - head/sys/kern

[Colin Percival]

Author: cperciva
Date: Sun Dec 31 09:22:31 2017
New Revision: 327426
URL: https://svnweb.freebsd.org/changeset/base/327426

Log:
  Use the TSLOG framework to record entry/exit timestamps for machine
  independent functions with important roles in the early boot process:
  mi_startup (with the "exit" recorded when it becomes swapper),
  start_init (with the "exit" recorded when the thread is about to
  "return" into the newly created init process), vfs_mountroot, and
  vfs_mountroot_wait.

Modified:
  head/sys/kern/init_main.c
  head/sys/kern/vfs_mountroot.c

Modified: head/sys/kern/init_main.c
==
--- head/sys/kern/init_main.c   Sun Dec 31 09:22:07 2017(r327425)
+++ head/sys/kern/init_main.c   Sun Dec 31 09:22:31 2017(r327426)
@@ -220,6 +220,8 @@ mi_startup(void)
int verbose;
 #endif
 
+   TSENTER();
+
if (boothowto & RB_VERBOSE)
bootverbose++;
 
@@ -313,6 +315,8 @@ restart:
}
}
 
+   TSEXIT();   /* Here so we don't overlap with start_init. */
+
mtx_assert(, MA_OWNED | MA_NOTRECURSED);
mtx_unlock();
 
@@ -706,6 +710,8 @@ start_init(void *dummy)
 
GIANT_REQUIRED;
 
+   TSENTER();  /* Here so we don't overlap with mi_startup. */
+
td = curthread;
p = td->td_proc;
 
@@ -799,6 +805,7 @@ start_init(void *dummy)
 */
if ((error = sys_execve(td, )) == EJUSTRETURN) {
mtx_unlock();
+   TSEXIT();
return;
}
if (error != ENOENT)

Modified: head/sys/kern/vfs_mountroot.c
==
--- head/sys/kern/vfs_mountroot.c   Sun Dec 31 09:22:07 2017
(r327425)
+++ head/sys/kern/vfs_mountroot.c   Sun Dec 31 09:22:31 2017
(r327426)
@@ -938,6 +938,8 @@ vfs_mountroot_wait(void)
struct timeval lastfail;
int curfail;
 
+   TSENTER();
+
curfail = 0;
while (1) {
DROP_GIANT();
@@ -957,6 +959,8 @@ vfs_mountroot_wait(void)
msleep(_holds, _holds_mtx, PZERO | PDROP, "roothold",
hz);
}
+
+   TSEXIT();
 }
 
 static int
@@ -1013,6 +1017,8 @@ vfs_mountroot(void)
struct thread *td;
time_t timebase;
int error;
+   
+   TSENTER();
 
td = curthread;
 
@@ -1062,6 +1068,8 @@ vfs_mountroot(void)
mtx_unlock(_holds_mtx);
 
EVENTHANDLER_INVOKE(mountroot);
+
+   TSEXIT();
 }
 
 static struct mntarg *
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r327425 - head/sys/amd64/amd64

[Colin Percival]

Author: cperciva
Date: Sun Dec 31 09:22:07 2017
New Revision: 327425
URL: https://svnweb.freebsd.org/changeset/base/327425

Log:
  Use the TSLOG framework to record entry/exit timestamps for hammer_time.
  The entry must be logged "manually" using TSRAW rather than TSENTER
  since PCPU data structures have not yet been initialized and thus
  curthread cannot be accessed;  is what will become curthread
  later in hammer_time.
  
  Other MD initialization code should be similarly instrumented in order
  to gain visibility into the time spent before entering mi_startup; this
  will require some care and testing from people with access to such
  hardware.

Modified:
  head/sys/amd64/amd64/machdep.c

Modified: head/sys/amd64/amd64/machdep.c
==
--- head/sys/amd64/amd64/machdep.c  Sun Dec 31 09:21:34 2017
(r327424)
+++ head/sys/amd64/amd64/machdep.c  Sun Dec 31 09:22:07 2017
(r327425)
@@ -1525,6 +1525,8 @@ hammer_time(u_int64_t modulep, u_int64_t physfree)
size_t kstack0_sz;
int late_console;
 
+   TSRAW(, TS_ENTER, __func__, NULL);
+
/*
 * This may be done better later if it gets more high level
 * components in it. If so just link td->td_proc here.
@@ -1773,6 +1775,8 @@ hammer_time(u_int64_t modulep, u_int64_t physfree)
x86_init_fdt();
 #endif
thread0.td_critnest = 0;
+
+   TSEXIT();
 
/* Location of kernel stack for locore */
return ((u_int64_t)thread0.td_pcb);
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r327423 - in head/sys: kern sys

[Colin Percival]

Author: cperciva
Date: Sun Dec 31 09:21:01 2017
New Revision: 327423
URL: https://svnweb.freebsd.org/changeset/base/327423

Log:
  Code for recording timestamps of events, especially function entries/exits.
  This is a very primitive system, intended for use in measuring performance
  during the early system boot, before more sophisticated tools like DTrace
  or infrastructure like kernel memory allocation and mutexes are available.
  
  Because this code records pointers to strings rather than copying strings
  (in order to keep the memory usage more manageable), if a kernel module is
  unloaded after logging an event, Bad Things can happen.  Users are advised
  to not do that.
  
  Since cycle counts from the early kernel boot are used as an initial entropy
  source, publishing this information to userland could result in inadequate
  entropy being kept private to the kernel RNG.  Users are advised to not
  enable this on systems with untrusted users.
  
  Discussed on: freebsd-current

Added:
  head/sys/kern/kern_tslog.c   (contents, props changed)
  head/sys/sys/tslog.h   (contents, props changed)

Added: head/sys/kern/kern_tslog.c
==
--- /dev/null   00:00:00 1970   (empty, because file is newly added)
+++ head/sys/kern/kern_tslog.c  Sun Dec 31 09:21:01 2017(r327423)
@@ -0,0 +1,118 @@
+/*-
+ * Copyright (c) 2017 Colin Percival
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in the
+ *documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include 
+__FBSDID("$FreeBSD$");
+
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include 
+#include 
+
+#ifndef TSLOGSIZE
+#define TSLOGSIZE 262144
+#endif
+
+static volatile long nrecs = 0;
+static struct timestamp {
+   void * td;
+   int type;
+   const char * f;
+   const char * s;
+   uint64_t tsc;
+} timestamps[TSLOGSIZE];
+
+void
+tslog(void * td, int type, const char * f, const char * s)
+{
+   uint64_t tsc = get_cyclecount();
+   long pos;
+
+   /* Grab a slot. */
+   pos = atomic_fetchadd_long(, 1);
+
+   /* Store record. */
+   if (pos < nitems(timestamps)) {
+   timestamps[pos].td = td;
+   timestamps[pos].type = type;
+   timestamps[pos].f = f;
+   timestamps[pos].s = s;
+   timestamps[pos].tsc = tsc;
+   }
+}
+
+static int
+sysctl_debug_tslog(SYSCTL_HANDLER_ARGS)
+{
+   int error;
+   struct sbuf *sb;
+   size_t i, limit;
+
+   /*
+* This code can race against the code in tslog() which stores
+* records: Theoretically we could end up reading a record after
+* its slots have been reserved but before it has been written.
+* Since this code takes orders of magnitude longer to run than
+* tslog() takes to write a record, it is highly unlikely that
+* anyone will ever experience this race.
+*/
+   sb = sbuf_new_for_sysctl(NULL, NULL, 1024, req);
+   limit = MIN(nrecs, nitems(timestamps));
+   for (i = 0; i < limit; i++) {
+   sbuf_printf(sb, "%p", timestamps[i].td);
+   sbuf_printf(sb, " %llu",
+   (unsigned long long)timestamps[i].tsc);
+   switch (timestamps[i].type) {
+   case TS_ENTER:
+   sbuf_printf(sb, " ENTER");
+   break;
+   case TS_EXIT:
+   sbuf_printf(sb, " EXIT");
+   break;
+   case TS_THREAD:
+   sbuf_printf(sb, " THREAD");
+   break;
+   case TS_EVENT:
+ 

svn commit: r327424 - head/sys/conf

[Colin Percival]

Author: cperciva
Date: Sun Dec 31 09:21:34 2017
New Revision: 327424
URL: https://svnweb.freebsd.org/changeset/base/327424

Log:
  Connect kern_tslog.c to the build and add TSLOG / TSLOGSIZE kernel options.
  These are intended for debugging purposes and should not be added to
  "generic" kernel configurations since they result in a nontrivial amount
  of memory being set aside for this purpose, can break if kernel modules are
  unloaded, and can potentially leak a dangerous amount of information about
  timestamps used as a source of kernel entropy.

Modified:
  head/sys/conf/NOTES
  head/sys/conf/files
  head/sys/conf/options

Modified: head/sys/conf/NOTES
==
--- head/sys/conf/NOTES Sun Dec 31 09:21:01 2017(r327423)
+++ head/sys/conf/NOTES Sun Dec 31 09:21:34 2017(r327424)
@@ -590,6 +590,20 @@ optionsSTACK
 #
 optionsNUM_CORE_FILES=5
 
+#
+# The TSLOG option enables timestamped logging of events, especially
+# function entries/exits, in order to track the time spent by the kernel.
+# In particular, this is useful when investigating the early boot process,
+# before it is possible to use more sophisticated tools like DTrace.
+# The TSLOGSIZE option controls the size of the (preallocated, fixed
+# length) buffer used for storing these events (default: 262144 records).
+#
+# For security reasons the TSLOG option should not be enabled on systems
+# used in production.
+#
+optionsTSLOG
+optionsTSLOGSIZE=262144
+
 
 #
 # PERFORMANCE MONITORING OPTIONS

Modified: head/sys/conf/files
==
--- head/sys/conf/files Sun Dec 31 09:21:01 2017(r327423)
+++ head/sys/conf/files Sun Dec 31 09:21:34 2017(r327424)
@@ -3802,6 +3802,7 @@ kern/kern_thr.c   standard
 kern/kern_thread.c standard
 kern/kern_time.c   standard
 kern/kern_timeout.cstandard
+kern/kern_tslog.c  optional tslog
 kern/kern_umtx.c   standard
 kern/kern_uuid.c   standard
 kern/kern_xxx.cstandard

Modified: head/sys/conf/options
==
--- head/sys/conf/options   Sun Dec 31 09:21:01 2017(r327423)
+++ head/sys/conf/options   Sun Dec 31 09:21:34 2017(r327424)
@@ -67,6 +67,8 @@ EARLY_PRINTF  opt_global.h
 TEXTDUMP_PREFERRED opt_ddb.h
 TEXTDUMP_VERBOSE   opt_ddb.h
 NUM_CORE_FILES opt_global.h
+TSLOG  opt_global.h
+TSLOGSIZE  opt_global.h
 
 # Miscellaneous options.
 ALQ
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r327120 - in head/sys: isa x86/isa

[Colin Percival]

On 12/23/17 14:57, Warner Losh wrote:
> - device_probe_and_attach(child);
> + err = device_probe_and_attach(child);
> + if (err == 0 && idev->id_vendorid == 0 &&
> + strcmp(kern_ident, "GENERIC") == 0)
> + device_printf(child,
> + "non-PNP ISA device will be removed from GENERIC in 
> FreeBSD 12.");

Maybe worth making this strncmp(kern_ident, "GENERIC", 7) so that it will
catch GENERIC-NODEBUG as well?

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r325841 - in head/sys: conf dev/mlx4 dev/mlx4/mlx4_core dev/mlx4/mlx4_en dev/mlx4/mlx4_ib modules/mlx4

[Colin Percival]

On 12/18/17 00:15, Hans Petter Selasky wrote:
> On 12/18/17 05:29, Colin Percival wrote:
>> Also, it breaks some work I have in progress for instrumenting SYSINITs.
>> Would you mind moving the DEFINE_MUTEX line to occur immediately prior to
>> the set_port_type function, rather than being placed inside it?
> 
> I'll have a look at this later today. Your point is valid!

On further examination, it looks like DEFINE_MUTEX is something used in Linux
kernel code, and the way it works there does allow it to be used inside a
function.  Is it possible to change the linuxkpi code to make it safe?  (It
looks like our mutex initialization is considerably more complicated than
what Linux does, so maybe not...?)

I have a feeling that we probably don't want to end up in a position of
"every time we import code from Linux, we need to grep for DEFINE_MUTEX
and hoist all of them out of functions".

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r325841 - in head/sys: conf dev/mlx4 dev/mlx4/mlx4_core dev/mlx4/mlx4_en dev/mlx4/mlx4_ib modules/mlx4

[Colin Percival]

On 11/15/17 03:14, Hans Petter Selasky wrote:
> Author: hselasky
> Date: Wed Nov 15 11:14:39 2017
> New Revision: 325841
> URL: https://svnweb.freebsd.org/changeset/base/325841
> 
> Log:
>   Update the mlx4 core and mlx4en(4) modules towards Linux v4.9.

sys/dev/mlx4/mlx4_core/mlx4_main.c, around line 1170:
> +static ssize_t set_port_type(struct device *dev,
> +  struct device_attribute *attr,
> +  const char *buf, size_t count)
> +{
> + struct mlx4_port_info *info = container_of(attr, struct mlx4_port_info,
> +port_attr);
> + struct mlx4_dev *mdev = info->dev;
> + enum mlx4_port_type port_type;
> + static DEFINE_MUTEX(set_port_type_mutex);

I'm not sure this is what you meant to do.  In particular, DEFINE_MUTEX uses
SX_SYSINIT_FLAGS, which creates a SYSINIT and a SYSUNINIT to initialize and
destroy the mutex -- meaning that it's being accessed from outside of the
function scope.  This seems to work because function-scope static variables
are compiled as if they have file scope, but I'm not at all convinced that
it's legal C.

Also, it breaks some work I have in progress for instrumenting SYSINITs.
Would you mind moving the DEFINE_MUTEX line to occur immediately prior to
the set_port_type function, rather than being placed inside it?

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r326565 - head/release/tools

[Colin Percival]

Author: cperciva
Date: Tue Dec  5 09:22:14 2017
New Revision: 326565
URL: https://svnweb.freebsd.org/changeset/base/326565

Log:
  Make EC2 instances use Amazon's NTP service for time synchronization.
  
  Since Amazon provides NTP servers within their network, this should
  be far superior to using the default NTP pools; and since the service
  is provided by Amazon there's very little risk in enabling it by
  default.  (If someone is able to compromise Amazon's NTP servers and
  exploit them to attack EC2 instances, they would almost certainly be
  able to compromise EC2 instances even without ntpd running...)
  
  MFC after:1 week
  Relnotes: EC2 instances now keep their clocks synchronized using
the Amazon Time Sync Service (aka. NTP).

Modified:
  head/release/tools/ec2.conf

Modified: head/release/tools/ec2.conf
==
--- head/release/tools/ec2.conf Tue Dec  5 09:08:48 2017(r326564)
+++ head/release/tools/ec2.conf Tue Dec  5 09:22:14 2017(r326565)
@@ -9,7 +9,7 @@
 export VM_EXTRA_PACKAGES="ec2-scripts firstboot-freebsd-update firstboot-pkgs 
dual-dhclient amazon-ssm-agent"
 
 # Set to a list of third-party software to enable in rc.conf(5).
-export VM_RC_LIST="ec2_configinit ec2_fetchkey ec2_ephemeralswap 
ec2_loghostkey firstboot_freebsd_update firstboot_pkgs"
+export VM_RC_LIST="ec2_configinit ec2_fetchkey ec2_ephemeralswap 
ec2_loghostkey firstboot_freebsd_update firstboot_pkgs ntpd"
 
 # Build with a 3 GB UFS partition; the growfs rc.d script will expand
 # the partition to fill the root disk after the EC2 instance is launched.
@@ -86,6 +86,11 @@ vm_extra_pre_umount() {
sed -i '' -e \
's/^#ChallengeResponseAuthentication 
yes/ChallengeResponseAuthentication no/' \
${DESTDIR}/etc/ssh/sshd_config
+
+   # Use the NTP service provided by Amazon
+   sed -i '' -e 's/^pool/#pool/' \
+   -e 's/^#server.*/server 169.254.169.123 iburst/' \
+   ${DESTDIR}/etc/ntp.conf
 
# The first time the AMI boots, the installed "first boot" scripts
# should be allowed to run:
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r326564 - head/release/tools

[Colin Percival]

Author: cperciva
Date: Tue Dec  5 09:08:48 2017
New Revision: 326564
URL: https://svnweb.freebsd.org/changeset/base/326564

Log:
  Resurrect r321659: Turn off ChallengeResponseAuthentication for EC2 AMIs.
  
  EC2 instances are normally launched with an SSH public key specified,
  which is then used for logging in (by default, as 'ec2-user').  Having
  ChallengeResponseAuthentication enabled (as FreeBSD's default sshd_config
  does) has no functional effect in a new EC2 instance, since you can't log
  in using a password until a password has been set -- but having this
  enabled results in alerts from automated scanning tools which can detect
  that sshd advertises support for keyboard-interactive logins (since they
  can't detect that accounts have no password set).
  
  EC2 users who want to use passwords to log in to their instances will need
  to set 'ChallengeResponseAuthentication yes' in FreeBSD 12.0 and later.
  
  Discussed with:   gjb, gtetlow, emaste, des
  Requested by: Amazon
  X-MFC:No
  Relnotes: ChallengeResponseAuthentication is turned off by default in
Amazon EC2 AMIs.

Modified:
  head/release/tools/ec2.conf

Modified: head/release/tools/ec2.conf
==
--- head/release/tools/ec2.conf Tue Dec  5 08:25:17 2017(r326563)
+++ head/release/tools/ec2.conf Tue Dec  5 09:08:48 2017(r326564)
@@ -81,6 +81,12 @@ vm_extra_pre_umount() {
# Load the kernel module for the Amazon "Elastic Network Adapter"
echo 'if_ena_load="YES"' >> ${DESTDIR}/boot/loader.conf
 
+   # Disable ChallengeResponseAuthentication according to EC2
+   # requirements.
+   sed -i '' -e \
+   's/^#ChallengeResponseAuthentication 
yes/ChallengeResponseAuthentication no/' \
+   ${DESTDIR}/etc/ssh/sshd_config
+
# The first time the AMI boots, the installed "first boot" scripts
# should be allowed to run:
# * ec2_configinit (download and process EC2 user-data)
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r325254 - head/release/tools

[Colin Percival]

Author: cperciva
Date: Wed Nov  1 00:33:54 2017
New Revision: 325254
URL: https://svnweb.freebsd.org/changeset/base/325254

Log:
  Add the amazon-ssm-agent package to EC2 AMI builds.  This makes it
  immediately available on instances which are running without internet
  access (or which can't rely on firstboot_pkgs to install it for some
  other reason).
  
  Note that this agent is not enabled by default; to enable it, add
  amazon_ssm_agent_enable="YES" to /etc/rc.conf, e.g., by placing the lines
>>/etc/rc.conf
amazon_ssm_agent_enable="YES"
  into the EC2 user-data.  In addition to being enabled, the agent requires
  keys to be provided via IAM Roles; users are encouraged to be very careful
  in using this functionality due to the inherent vulnerability in the idea
  of providing credentials via a service accessible to any process which can
  open an HTTP connection.
  
  Requested by: Amazon
  No objection from:re@
  Relnotes: FreeBSD/EC2 AMIs now include the Amazon EC2 Systems Manager
(SSM) Agent.

Modified:
  head/release/tools/ec2.conf

Modified: head/release/tools/ec2.conf
==
--- head/release/tools/ec2.conf Tue Oct 31 23:48:57 2017(r325253)
+++ head/release/tools/ec2.conf Wed Nov  1 00:33:54 2017(r325254)
@@ -6,7 +6,7 @@
 # Packages to install into the image we're creating.  This is a deliberately
 # minimalist set, providing only the packages necessary to bootstrap further
 # package installation as specified via EC2 user-data.
-export VM_EXTRA_PACKAGES="ec2-scripts firstboot-freebsd-update firstboot-pkgs 
dual-dhclient"
+export VM_EXTRA_PACKAGES="ec2-scripts firstboot-freebsd-update firstboot-pkgs 
dual-dhclient amazon-ssm-agent"
 
 # Set to a list of third-party software to enable in rc.conf(5).
 export VM_RC_LIST="ec2_configinit ec2_fetchkey ec2_ephemeralswap 
ec2_loghostkey firstboot_freebsd_update firstboot_pkgs"
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r321985 - head/sys/ofed/drivers/infiniband/core

[Colin Percival]

On 08/03/17 23:28, Hans Petter Selasky wrote:
> On 08/03/17 16:37, Conrad Meyer wrote:
>> Is it not important that the subtraction and result are evaluated
>> without truncation?
> 
> ticks is a circular counter. Assuming time = 0 and jiffies = -1U, then "delay"
> becomes a very large negative value, because long is used, and the delay <= 0
> check, is no longer working like expected.
> 
> Casting to "int" or truncating is the right thing to do in this case.

Signed integer overflow is undefined.  Using 'int' is liable to cause problems
after 2^32 ticks.

Colin Percival

>>> Log:
>>>Ticks are 32-bit in FreeBSD.
>>>
>>>MFC after:3 days
>>>Sponsored by: Mellanox Technologies
>>>
>>> Modified:
>>>head/sys/ofed/drivers/infiniband/core/addr.c
>>>
>>> Modified: head/sys/ofed/drivers/infiniband/core/addr.c
>>> ==
>>> --- head/sys/ofed/drivers/infiniband/core/addr.cThu Aug  3 09:14:43
>>> 2017(r321984)
>>> +++ head/sys/ofed/drivers/infiniband/core/addr.cThu Aug  3 09:18:25
>>> 2017(r321985)
>>> @@ -187,10 +187,10 @@ EXPORT_SYMBOL(rdma_translate_ip);
>>>
>>>   static void set_timeout(unsigned long time)
>>>   {
>>> -   unsigned long delay;
>>> +   int delay;  /* under FreeBSD ticks are 32-bit */
>>>
>>>  delay = time - jiffies;
>>> -   if ((long)delay <= 0)
>>> +   if (delay <= 0)
>>>  delay = 1;
>>>
>>>  mod_delayed_work(addr_wq, , delay);

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r318441 - in head/etc: . cron.d

[Colin Percival]

On 06/03/17 21:39, Rodney W. Grimes wrote:
> Are we going to end up with /etc/rc.conf.d?

We've had this for over 15 years -- ever since NetBSD rc.d system was
imported.

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r319562 - head/usr.bin/factor

[Colin Percival]

Author: cperciva
Date: Sun Jun  4 02:45:18 2017
New Revision: 319562
URL: https://svnweb.freebsd.org/changeset/base/319562

Log:
  Delete obsolete paragraph; primes(6) is now able to list primes for the
  full range up to 2^64 - 1.
  
  MFC after:1 week

Modified:
  head/usr.bin/factor/factor.6

Modified: head/usr.bin/factor/factor.6
==
--- head/usr.bin/factor/factor.6Sun Jun  4 02:36:37 2017
(r319561)
+++ head/usr.bin/factor/factor.6Sun Jun  4 02:45:18 2017
(r319562)
@@ -119,9 +119,3 @@ cannot handle the
 factor list,
 .Nm primes
 will not get you a world record.
-.Pp
-.Nm primes
-is unable to list primes between 3825123056546413050 and 18446744073709551615
-since it relies on strong pseudoprime tests after sieving, and nobody has
-proven how many strong pseudoprime tests are required to prove primality for
-integers larger than 3825123056546413050.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r319561 - head/usr.bin/primes

[Colin Percival]

Author: cperciva
Date: Sun Jun  4 02:36:37 2017
New Revision: 319561
URL: https://svnweb.freebsd.org/changeset/base/319561

Log:
  Using results from
  J. Sorenson and J. Webster, Strong pseudoprimes to twelve prime
  bases, Math. Comp. 86(304):985-1003, 2017.
  teach primes(6) to enumerate primes up to 2^64 - 1.  Until Sorenson
  and Webster's paper, we did not know how many strong speudoprime tests
  were required when testing alleged primes between 3825123056546413051
  and 2^64 - 1.
  
  Reported by:  Luiz Henrique de Figueiredo
  Relnotes: primes(6) now enumerates primes beyond 3825123056546413050,
up to a new limit of 2^64 - 1.
  MFC After:1 week

Modified:
  head/usr.bin/primes/primes.c
  head/usr.bin/primes/primes.h
  head/usr.bin/primes/spsp.c

Modified: head/usr.bin/primes/primes.c
==
--- head/usr.bin/primes/primes.cSun Jun  4 02:21:38 2017
(r319560)
+++ head/usr.bin/primes/primes.cSun Jun  4 02:36:37 2017
(r319561)
@@ -120,7 +120,7 @@ main(int argc, char *argv[])
argv += optind;
 
start = 0;
-   stop = SPSPMAX;
+   stop = (uint64_t)(-1);
 
/*
 * Convert low and high args.  Strtoumax(3) sets errno to
@@ -147,8 +147,6 @@ main(int argc, char *argv[])
err(1, "%s", argv[1]);
if (*p != '\0')
errx(1, "%s: illegal numeric format.", argv[1]);
-   if (stop > SPSPMAX)
-   errx(1, "%s: stop value too large.", argv[1]);
break;
case 1:
/* Start on the command line. */

Modified: head/usr.bin/primes/primes.h
==
--- head/usr.bin/primes/primes.hSun Jun  4 02:21:38 2017
(r319560)
+++ head/usr.bin/primes/primes.hSun Jun  4 02:36:37 2017
(r319561)
@@ -72,6 +72,3 @@ extern const size_t pattern_size; /* length of pattern
 
 /* Test for primality using strong pseudoprime tests. */
 int isprime(ubig);
-
-/* Maximum value which the SPSP code can handle. */
-#defineSPSPMAX 3825123056546413050ULL

Modified: head/usr.bin/primes/spsp.c
==
--- head/usr.bin/primes/spsp.c  Sun Jun  4 02:21:38 2017(r319560)
+++ head/usr.bin/primes/spsp.c  Sun Jun  4 02:36:37 2017(r319561)
@@ -32,23 +32,32 @@ __FBSDID("$FreeBSD$");
 
 #include "primes.h"
 
-/* Return a * b % n, where 0 <= a, b < 2^63, 0 < n < 2^63. */
+/* Return a * b % n, where 0 < n. */
 static uint64_t
 mulmod(uint64_t a, uint64_t b, uint64_t n)
 {
uint64_t x = 0;
+   uint64_t an = a % n;
 
while (b != 0) {
-   if (b & 1)
-   x = (x + a) % n;
-   a = (a + a) % n;
+   if (b & 1) {
+   x += an;
+   if ((x < an) || (x >= n))
+   x -= n;
+   }
+   if (an + an < an)
+   an = an + an - n;
+   else if (an + an >= n)
+   an = an + an - n;
+   else
+   an = an + an;
b >>= 1;
}
 
return (x);
 }
 
-/* Return a^r % n, where 0 <= a < 2^63, 0 < n < 2^63. */
+/* Return a^r % n, where 0 < n. */
 static uint64_t
 powmod(uint64_t a, uint64_t r, uint64_t n)
 {
@@ -173,9 +182,20 @@ isprime(ubig _n)
if (n < 3825123056546413051)
return (1);
 
-   /* We can't handle values larger than this. */
-   assert(n <= SPSPMAX);
+   /*
+* Value from:
+* J. Sorenson and J. Webster, Strong pseudoprimes to twelve prime
+* bases, Math. Comp. 86(304):985-1003, 2017.
+*/
 
-   /* UNREACHABLE */
-   return (0);
+   /* No SPSPs to bases 2..37 less than 318665857834031151167461. */
+   if (!spsp(n, 29))
+   return (0);
+   if (!spsp(n, 31))
+   return (0);
+   if (!spsp(n, 37))
+   return (0);
+
+   /* All 64-bit values are less than 318665857834031151167461. */
+   return (1);
 }
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r319491 - head/sys/dev/xen/netfront

[Colin Percival]

On 06/02/17 03:29, Roger Pau Monn� wrote:
> On Fri, Jun 02, 2017 at 07:03:31AM +0000, Colin Percival wrote:
>>   Skip setting the MTU in the netfront driver (xn# devices) if the new MTU
>>   is the same as the old MTU. [...]
>>   Maintainers of other network interface drivers may wish to consider making
>>   the corresponding change; the handling of SIOCSIFMTU does not seem to
>>   exhibit a great deal of consistency between drivers.
> 
> Is there any reason this check (ifp->if_mtu == ifr->ifr_mtu) is not
> done at a higher level for all the drivers? It seems pointless to add
> this chunk everywhere.

I wondered about that.  Some drivers already do this (if_sk, if_age, if_jme,
if_tx, if_bge, if_vge, ...) but every driver seems to spell it differently;
and many drivers do things beyond merely recording the new value and
re-initializing.

I certainly have no objection to seeing a more generic handling of interface
MTU setting, but I don't know enough about the network stack to do this
myself.

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r319491 - head/sys/dev/xen/netfront

[Colin Percival]

Author: cperciva
Date: Fri Jun  2 07:03:31 2017
New Revision: 319491
URL: https://svnweb.freebsd.org/changeset/base/319491

Log:
  Skip setting the MTU in the netfront driver (xn# devices) if the new MTU
  is the same as the old MTU.  In particular, on Amazon EC2 "T2" instances
  without this change, the network interface is reinitialized every 30
  minutes due to the MTU being (re)set when a new DHCP lease is obtained,
  causing packets to be dropped, along with annoying syslog messages about
  the link state changing.
  
  As a side note, the behaviour this commit fixes was responsible for
  exposing the locking problems fixed via r318523 and r318631.
  
  Maintainers of other network interface drivers may wish to consider making
  the corresponding change; the handling of SIOCSIFMTU does not seem to
  exhibit a great deal of consistency between drivers.
  
  MFC after:1 week

Modified:
  head/sys/dev/xen/netfront/netfront.c

Modified: head/sys/dev/xen/netfront/netfront.c
==
--- head/sys/dev/xen/netfront/netfront.cFri Jun  2 03:53:34 2017
(r319490)
+++ head/sys/dev/xen/netfront/netfront.cFri Jun  2 07:03:31 2017
(r319491)
@@ -1767,6 +1767,9 @@ xn_ioctl(struct ifnet *ifp, u_long cmd, caddr_t data)
 #endif
break;
case SIOCSIFMTU:
+   if (ifp->if_mtu == ifr->ifr_mtu)
+   break;
+
ifp->if_mtu = ifr->ifr_mtu;
ifp->if_drv_flags &= ~IFF_DRV_RUNNING;
xn_ifinit(sc);
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r318894 - in head/release: . tools

[Colin Percival]

Author: cperciva
Date: Thu May 25 19:02:54 2017
New Revision: 318894
URL: https://svnweb.freebsd.org/changeset/base/318894

Log:
  Turn on support for the Amazon "Elastic Network Adapter" in EC2 AMIs.
  
  X-MFC-after:  318647 + fixes for some lock ordering warnings

Modified:
  head/release/Makefile.ec2
  head/release/tools/ec2.conf

Modified: head/release/Makefile.ec2
==
--- head/release/Makefile.ec2   Thu May 25 18:55:10 2017(r318893)
+++ head/release/Makefile.ec2   Thu May 25 19:02:54 2017(r318894)
@@ -51,7 +51,7 @@ ec2ami: cw-ec2 ${CW_EC2_PORTINSTALL}
@echo "--"
@false
 .endif
-   /usr/local/bin/bsdec2-image-upload ${PUBLISH} --sriov \
+   /usr/local/bin/bsdec2-image-upload ${PUBLISH} --sriov --ena \
${.OBJDIR}/ec2.raw \
"${TYPE} ${REVISION}-${BRANCH}-${TARGET}${AMINAMESUFFIX}" \
"${TYPE} ${REVISION}-${BRANCH}-${TARGET}" \

Modified: head/release/tools/ec2.conf
==
--- head/release/tools/ec2.conf Thu May 25 18:55:10 2017(r318893)
+++ head/release/tools/ec2.conf Thu May 25 19:02:54 2017(r318894)
@@ -78,6 +78,9 @@ vm_extra_pre_umount() {
# nodes, but apply the workaround just in case.
echo 'hw.broken_txfifo="1"' >> ${DESTDIR}/boot/loader.conf
 
+   # Load the kernel module for the Amazon "Elastic Network Adapter"
+   echo 'if_ena_load="YES"' >> ${DESTDIR}/boot/loader.conf
+
# The first time the AMI boots, the installed "first boot" scripts
# should be allowed to run:
# * ec2_configinit (download and process EC2 user-data)
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r318757 - head

[Colin Percival]

On 05/23/17 14:39, Allan Jude wrote:
> On 2017-05-23 17:33, Alexey Dokuchaev wrote:
>> On Tue, May 23, 2017 at 08:25:49PM +, Ed Maste wrote:
>>> New Revision: 318757
>>> URL: https://svnweb.freebsd.org/changeset/base/318757
>>>
>>> Log:
>>>   Add note to UPDATING for ino64 to follow the standard upgrade process
>>> ...
>>> +20170523:
>>> +   The "ino64" 64-bit inode project has been committed, which extends
>>> +   a number of types to 64 bits.
>>
>> For the dumb people among us, what is it all about?  E.g., what's so cool
>> about it?
> 
> 64bit inodes allow you to have more than 4 billion files in one file
> system, or NFS mount.

It can avoid problems on NFS even if you have less than 2^32 files, since
NFS uses 64-bit fileids and is not required to use the values 0, 1, 2, 3...
for files.  In Amazon EFS for example every fileid is > 2^32 and there's no
guarantee that two files won't map to the same 32-bit inode # (pre-ino64,
we simply reduced the fileid mod 2^32 to generate an inode #).

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r301198 - head/sys/dev/xen/netfront

[Colin Percival]

On 05/09/17 03:09, Roger Pau Monn� wrote:
> On Wed, May 03, 2017 at 05:13:40AM +0000, Colin Percival wrote:
>> On 06/02/16 04:16, Roger Pau Monn� wrote:
>>> Author: royger
>>> Date: Thu Jun  2 11:16:35 2016
>>> New Revision: 301198
>>> URL: https://svnweb.freebsd.org/changeset/base/301198
>>
>> I think this commit is responsible for panics I'm seeing in EC2 on T2 family
>> instances.  [...]
>> but under high traffic volumes I think a separate thread can already be
>> running in xn_rxeof, having dropped the RX lock while it passes a packet
>> up the stack.  This would result in two different threads trying to process
>> the same set of responses from the ring, with (unsurprisingly) bad results.
> 
> Hm, right, xn_rxeof drops the lock while pushing the packet up the stack.
> There's a "XXX" comment on top of that, could you try to remove the lock
> dripping and see what happens?
> 
> I'm not sure there's any reason to drop the lock here, I very much doubt
> if_input is going to sleep.

Judging by
$ grep -R -B 1 -A 1 if_input /usr/src/sys/dev
I'm pretty sure that we do indeed need to drop the lock.  If it's possible
to enter if_input while holding locks, there are a *lot* of network interface
drivers which are dropping locks unnecessarily...

>> 3. Why xn_ifinit_locked is consuming ring responses.
> 
> There might be pending RX packets on the ring, so netfront consumes them and
> signals netback. In the unlikely event that the RX ring was full when
> xn_ifinit_locked is called, not doing this would mean the RX queue would get
> stuck forever, since there's no guarantee netfront will receive event channel
> notifications.

In that case, I'm guessing it would be safe to skip this if another thread is
already running xn_rxeof and chewing through the packets on the ring?  It
would be easy to set a flag in xn_rxeof before we drop locks.

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r301198 - head/sys/dev/xen/netfront

[Colin Percival]

On 06/02/16 04:16, Roger Pau Monné wrote:
> Author: royger
> Date: Thu Jun  2 11:16:35 2016
> New Revision: 301198
> URL: https://svnweb.freebsd.org/changeset/base/301198

I think this commit is responsible for panics I'm seeing in EC2 on T2 family
instances.  Every time a DHCP request is made, we call into xn_ifinit_locked
(not sure why -- something to do with making the interface promiscuous?) and
hit this code

> @@ -1760,7 +1715,7 @@ xn_ifinit_locked(struct netfront_info *n
>   xn_alloc_rx_buffers(rxq);
>   rxq->ring.sring->rsp_event = rxq->ring.rsp_cons + 1;
>   if (RING_HAS_UNCONSUMED_RESPONSES(>ring))
> - taskqueue_enqueue(rxq->tq, >intrtask);
> + xn_rxeof(rxq);
>   XN_RX_UNLOCK(rxq);
>   }

but under high traffic volumes I think a separate thread can already be
running in xn_rxeof, having dropped the RX lock while it passes a packet
up the stack.  This would result in two different threads trying to process
the same set of responses from the ring, with (unsurprisingly) bad results.

I'm not 100% sure that this is what's causing the panic, but it's definitely
happening under high traffic conditions immediately after xn_ifinit_locked is
called, so I think my speculation is well-founded.

There are a few things I don't understand here:
1. Why DHCP requests are resulting in calls into xn_ifinit_locked.
2. Why the calls into xn_ifinit_locked are only happening on T2 instances
and not on any of the other EC2 instances I've tried.
3. Why xn_ifinit_locked is consuming ring responses.
so I'm not sure what the solution is, but hopefully someone who knows this
code better will be able to help...

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r317277 - head/sys/crypto/chacha20

[Colin Percival]

On 04/21/17 19:33, Rui Paulo wrote:
> On Apr 21, 2017, at 18:06, Dag-Erling Smørgrav <d...@freebsd.org> wrote:
>> Author: des
>> Date: Sat Apr 22 01:06:23 2017
>> New Revision: 317277
>> URL: https://svnweb.freebsd.org/changeset/base/317277
>>
>> Log:
>>  Fix counter increment in Salsa and ChaCha.
>>
>>  In my eagerness to eliminate a branch which is taken once per 2^38
>>  bytes of keystream, I forgot that the state words are in host order.
>>  Thus, the counter increment code worked fine on little-endian
>>  machines, but not on big-endian ones.  Switch to a simpler (branchful)
>>  solution.
> 
> I’m surprised there’s no mention of who reviewed your change, especially when 
> you’re changing crypto code.

Reviewed by:cperciva

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r312214 - head/release/tools

[Colin Percival]

Author: cperciva
Date: Sun Jan 15 09:06:45 2017
New Revision: 312214
URL: https://svnweb.freebsd.org/changeset/base/312214

Log:
  Enable IPv6 networking on Amazon EC2.
  
  MFC after:1 week

Modified:
  head/release/tools/ec2.conf

Modified: head/release/tools/ec2.conf
==
--- head/release/tools/ec2.conf Sun Jan 15 09:05:26 2017(r312213)
+++ head/release/tools/ec2.conf Sun Jan 15 09:06:45 2017(r312214)
@@ -6,7 +6,7 @@
 # Packages to install into the image we're creating.  This is a deliberately
 # minimalist set, providing only the packages necessary to bootstrap further
 # package installation as specified via EC2 user-data.
-export VM_EXTRA_PACKAGES="ec2-scripts firstboot-freebsd-update firstboot-pkgs"
+export VM_EXTRA_PACKAGES="ec2-scripts firstboot-freebsd-update firstboot-pkgs 
dual-dhclient"
 
 # Set to a list of third-party software to enable in rc.conf(5).
 export VM_RC_LIST="ec2_configinit ec2_fetchkey ec2_ephemeralswap 
ec2_loghostkey firstboot_freebsd_update firstboot_pkgs"
@@ -39,8 +39,9 @@ vm_extra_pre_umount() {
# time; expand our filesystem to fill the disk.
echo 'growfs_enable="YES"' >> ${DESTDIR}/etc/rc.conf
 
-   # EC2 instances use DHCP to get their network configuration.
-   echo 'ifconfig_DEFAULT="SYNCDHCP"' >> ${DESTDIR}/etc/rc.conf
+   # EC2 instances use DHCP to get their network configuration.  IPv6
+   # requires accept_rtadv.
+   echo 'ifconfig_DEFAULT="SYNCDHCP accept_rtadv"' >> 
${DESTDIR}/etc/rc.conf
 
# Unless the system has been configured via EC2 user-data, the user
# will need to SSH in to do anything.
@@ -51,6 +52,10 @@ vm_extra_pre_umount() {
# via EC2 user-data.
echo 'firstboot_pkgs_list="awscli"' >> ${DESTDIR}/etc/rc.conf
 
+   # Enable IPv6 on all interfaces, and use DHCP on both IPv4 and IPv6.
+   echo 'ipv6_activate_all_interfaces="YES"' >> ${DESTDIR}/etc/rc.conf
+   echo 'dhclient_program="/usr/local/sbin/dual-dhclient"' >> 
${DESTDIR}/etc/rc.conf
+
# The EC2 console is output-only, so while printing a backtrace can
# be useful, there's no point dropping into a debugger or waiting
# for a keypress.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r311945 - head/usr.bin/fortune/fortune

[Colin Percival]

Author: cperciva
Date: Wed Jan 11 22:50:57 2017
New Revision: 311945
URL: https://svnweb.freebsd.org/changeset/base/311945

Log:
  Remove obsolete path from fortune(6).  This was inadvertantly left over
  when fortune and other games moved from /usr/games to /usr/bin; I am
  removing rather than correcting it since we normally do not mention in
  the FILES section the paths to programs in /usr/bin/.
  
  PR:   215962
  Reported by:  Andras Farkas

Modified:
  head/usr.bin/fortune/fortune/fortune.6

Modified: head/usr.bin/fortune/fortune/fortune.6
==
--- head/usr.bin/fortune/fortune/fortune.6  Wed Jan 11 22:10:56 2017
(r311944)
+++ head/usr.bin/fortune/fortune/fortune.6  Wed Jan 11 22:50:57 2017
(r311945)
@@ -176,7 +176,6 @@ it was up to on disk.
 .El
 .Sh FILES
 .Bl -tag -width ".Pa /usr/share/games/fortune/*"
-.It Pa /usr/games/fortune
 .It Pa /usr/share/games/fortune/*
 the fortunes databases (those files ending
 .Dq Pa -o
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r310179 - head/usr.sbin/portsnap/portsnap

[Colin Percival]

Author: cperciva
Date: Fri Dec 16 22:37:16 2016
New Revision: 310179
URL: https://svnweb.freebsd.org/changeset/base/310179

Log:
  Avoid division by zero in the rare case that portsnap needs to fetch
  zero patches.  (This avoids two "dc: divide by zero" warnings.)
  
  MFC after:3 days

Modified:
  head/usr.sbin/portsnap/portsnap/portsnap.sh

Modified: head/usr.sbin/portsnap/portsnap/portsnap.sh
==
--- head/usr.sbin/portsnap/portsnap/portsnap.sh Fri Dec 16 21:58:48 2016
(r310178)
+++ head/usr.sbin/portsnap/portsnap/portsnap.sh Fri Dec 16 22:37:16 2016
(r310179)
@@ -619,8 +619,10 @@ fetch_progress() {
 
 pct_fmt()
 {
-   printf " \r"
-   printf "($1/$2) %02.2f%% " `echo "scale=4;$LNC / $TOTAL * 100"|bc`
+   if [ $TOTAL -gt 0 ]; then
+   printf " \r"
+   printf "($1/$2) %02.2f%% " `echo "scale=4;$LNC / $TOTAL * 
100"|bc`
+   fi
 }
 
 fetch_progress_percent() {
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r310086 - head/sys/dev/xen/blkfront

[Colin Percival]

On 12/14/16 11:28, Dimitry Andric wrote:
> Log:
>   In xbd_connect(), use correct scanf conversion specifiers for the
>   feature_barrier and feature_flush variables.  Otherwise, adjacent
>   variables on the stack, such as sector_size, may be overwritten, with
>   disastrous results.

Thanks!  Did you happen to notice what stack variable (if any?) was being
overwritten under clang 3.8.0?  Just wondering if there might be some
undiscovered issue lurking in FreeBSD releases which will cause other less
obvious problems.

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"