Re: svn commit: r367288 - head/sys/compat/linux

2020-11-02 Thread Conrad Meyer
No, not as far as I can tell.

On Mon, Nov 2, 2020 at 5:48 PM Andrew Gallatin  wrote:
>
> On 11/2/20 8:19 PM, Conrad Meyer wrote:
> >
> > Log:
> >linux(4): Emulate Linux SOL_SOCKET:SO_PASSCRED
> >
> >This is required by some major linux applications, such as Chrome and
> >Firefox.  (As well as Electron-using applications, which are essentially
> >a bundled version of Chrome.)
> >
>
> Awesome!  Does this get electron apps working?
>
> Drew
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


svn commit: r367290 - head/sys/compat/linux

2020-11-02 Thread Conrad Meyer
Author: cem
Date: Tue Nov  3 02:10:54 2020
New Revision: 367290
URL: https://svnweb.freebsd.org/changeset/base/367290

Log:
  linux(4) prctl(2): Implement PR_[GS]ET_DUMPABLE
  
  Proxy the flag to the roughly analogous FreeBSD procctl 'TRACE'.
  
  TRACE-disabled processes are not coredumped, and Linux !DUMPABLE processes
  can not be ptraced.  There are some additional semantics around ownership of
  files in the /proc/[pid] pseudo-filesystem, which we do not attempt to
  emulate correctly at this time.
  
  Reviewed by:  markj (earlier version)
  Differential Revision:https://reviews.freebsd.org/D27015

Modified:
  head/sys/compat/linux/linux_misc.c
  head/sys/compat/linux/linux_misc.h

Modified: head/sys/compat/linux/linux_misc.c
==
--- head/sys/compat/linux/linux_misc.c  Tue Nov  3 01:38:16 2020
(r367289)
+++ head/sys/compat/linux/linux_misc.c  Tue Nov  3 02:10:54 2020
(r367290)
@@ -1937,7 +1937,7 @@ linux_prctl(struct thread *td, struct linux_prctl_args
int error = 0, max_size;
struct proc *p = td->td_proc;
char comm[LINUX_MAX_COMM_LEN];
-   int pdeath_signal;
+   int pdeath_signal, trace_state;
 
switch (args->option) {
case LINUX_PR_SET_PDEATHSIG:
@@ -1955,10 +1955,46 @@ linux_prctl(struct thread *td, struct linux_prctl_args
return (copyout(_signal,
(void *)(register_t)args->arg2,
sizeof(pdeath_signal)));
+   /*
+* In Linux, this flag controls if set[gu]id processes can coredump.
+* There are additional semantics imposed on processes that cannot
+* coredump:
+* - Such processes can not be ptraced.
+* - There are some semantics around ownership of process-related files
+*   in the /proc namespace.
+*
+* In FreeBSD, we can (and by default, do) disable setuid coredump
+* system-wide with 'sugid_coredump.'  We control tracability on a
+* per-process basis with the procctl PROC_TRACE (=> P2_NOTRACE flag).
+* By happy coincidence, P2_NOTRACE also prevents coredumping.  So the
+* procctl is roughly analogous to Linux's DUMPABLE.
+*
+* So, proxy these knobs to the corresponding PROC_TRACE setting.
+*/
+   case LINUX_PR_GET_DUMPABLE:
+   error = kern_procctl(td, P_PID, p->p_pid, PROC_TRACE_STATUS,
+   _state);
+   if (error != 0)
+   return (error);
+   td->td_retval[0] = (trace_state != -1);
+   return (0);
case LINUX_PR_SET_DUMPABLE:
-   linux_msg(td, "unsupported prctl PR_SET_DUMPABLE");
-   error = EINVAL;
-   break;
+   /*
+* It is only valid for userspace to set one of these two
+* flags, and only one at a time.
+*/
+   switch (args->arg2) {
+   case LINUX_SUID_DUMP_DISABLE:
+   trace_state = PROC_TRACE_CTL_DISABLE_EXEC;
+   break;
+   case LINUX_SUID_DUMP_USER:
+   trace_state = PROC_TRACE_CTL_ENABLE;
+   break;
+   default:
+   return (EINVAL);
+   }
+   return (kern_procctl(td, P_PID, p->p_pid, PROC_TRACE_CTL,
+   _state));
case LINUX_PR_GET_KEEPCAPS:
/*
 * Indicate that we always clear the effective and

Modified: head/sys/compat/linux/linux_misc.h
==
--- head/sys/compat/linux/linux_misc.h  Tue Nov  3 01:38:16 2020
(r367289)
+++ head/sys/compat/linux/linux_misc.h  Tue Nov  3 02:10:54 2020
(r367290)
@@ -50,6 +50,7 @@
 * Second arg is a ptr to return the
 * signal.
 */
+#defineLINUX_PR_GET_DUMPABLE   3
 #defineLINUX_PR_SET_DUMPABLE   4
 #defineLINUX_PR_GET_KEEPCAPS   7   /* Get drop capabilities on 
setuid */
 #defineLINUX_PR_SET_KEEPCAPS   8   /* Set drop capabilities on 
setuid */
@@ -61,6 +62,11 @@
 #defineLINUX_PR_SET_PTRACER1499557217
 
 #defineLINUX_MAX_COMM_LEN  16  /* Maximum length of the 
process name. */
+
+/* For GET/SET DUMPABLE */
+#defineLINUX_SUID_DUMP_DISABLE 0   /* Don't coredump setuid 
processes. */
+#defineLINUX_SUID_DUMP_USER1   /* Dump as user of process. */
+#defineLINUX_SUID_DUMP_ROOT2   /* Dump as root. */
 
 #defineLINUX_MREMAP_MAYMOVE1
 #defineLINUX_MREMAP_FIXED  2
___
svn-src-head@freebsd.org mailing list

Re: svn commit: r367288 - head/sys/compat/linux

2020-11-02 Thread Andrew Gallatin

On 11/2/20 8:19 PM, Conrad Meyer wrote:


Log:
   linux(4): Emulate Linux SOL_SOCKET:SO_PASSCRED
   
   This is required by some major linux applications, such as Chrome and

   Firefox.  (As well as Electron-using applications, which are essentially
   a bundled version of Chrome.)
   


Awesome!  Does this get electron apps working?

Drew
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


svn commit: r367288 - head/sys/compat/linux

2020-11-02 Thread Conrad Meyer
Author: cem
Date: Tue Nov  3 01:19:13 2020
New Revision: 367288
URL: https://svnweb.freebsd.org/changeset/base/367288

Log:
  linux(4): Emulate Linux SOL_SOCKET:SO_PASSCRED
  
  This is required by some major linux applications, such as Chrome and
  Firefox.  (As well as Electron-using applications, which are essentially
  a bundled version of Chrome.)
  
  Reviewed by:  markj
  Differential Revision:https://reviews.freebsd.org/D27012

Modified:
  head/sys/compat/linux/linux_socket.c

Modified: head/sys/compat/linux/linux_socket.c
==
--- head/sys/compat/linux/linux_socket.cTue Nov  3 01:17:45 2020
(r367287)
+++ head/sys/compat/linux/linux_socket.cTue Nov  3 01:19:13 2020
(r367288)
@@ -222,6 +222,8 @@ linux_to_bsd_so_sockopt(int opt)
return (SO_LINGER);
case LINUX_SO_REUSEPORT:
return (SO_REUSEPORT_LB);
+   case LINUX_SO_PASSCRED:
+   return (LOCAL_CREDS_PERSISTENT);
case LINUX_SO_PEERCRED:
return (LOCAL_PEERCRED);
case LINUX_SO_RCVLOWAT:
@@ -1445,6 +1447,9 @@ linux_setsockopt(struct thread *td, struct linux_setso
case SOL_SOCKET:
name = linux_to_bsd_so_sockopt(args->optname);
switch (name) {
+   case LOCAL_CREDS_PERSISTENT:
+   level = SOL_LOCAL;
+   break;
case SO_RCVTIMEO:
/* FALLTHROUGH */
case SO_SNDTIMEO:
@@ -1522,6 +1527,9 @@ linux_getsockopt(struct thread *td, struct linux_getso
case SOL_SOCKET:
name = linux_to_bsd_so_sockopt(args->optname);
switch (name) {
+   case LOCAL_CREDS_PERSISTENT:
+   level = SOL_LOCAL;
+   break;
case SO_RCVTIMEO:
/* FALLTHROUGH */
case SO_SNDTIMEO:
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


svn commit: r367287 - in head: share/man/man4 sys/kern sys/sys

2020-11-02 Thread Conrad Meyer
Author: cem
Date: Tue Nov  3 01:17:45 2020
New Revision: 367287
URL: https://svnweb.freebsd.org/changeset/base/367287

Log:
  unix(4): Add SOL_LOCAL:LOCAL_CREDS_PERSISTENT
  
  This option is intended to be semantically identical to Linux's
  SOL_SOCKET:SO_PASSCRED.  For now, it is mutually exclusive with the
  pre-existing sockopt SOL_LOCAL:LOCAL_CREDS.
  
  Reviewed by:  markj (penultimate version)
  Differential Revision:https://reviews.freebsd.org/D27011

Modified:
  head/share/man/man4/unix.4
  head/sys/kern/uipc_usrreq.c
  head/sys/sys/un.h
  head/sys/sys/unpcb.h

Modified: head/share/man/man4/unix.4
==
--- head/share/man/man4/unix.4  Tue Nov  3 01:10:27 2020(r367286)
+++ head/share/man/man4/unix.4  Tue Nov  3 01:17:45 2020(r367287)
@@ -28,7 +28,7 @@
 .\" @(#)unix.4 8.1 (Berkeley) 6/9/93
 .\" $FreeBSD$
 .\"
-.Dd August 3, 2020
+.Dd November 2, 2020
 .Dt UNIX 4
 .Os
 .Sh NAME
@@ -201,7 +201,7 @@ which can be set with
 .Xr setsockopt 2
 and tested with
 .Xr getsockopt 2 :
-.Bl -tag -width ".Dv LOCAL_CONNWAIT"
+.Bl -tag -width ".Dv LOCAL_CREDS_PERSISTENT"
 .It Dv LOCAL_CREDS
 This option may be enabled on
 .Dv SOCK_DGRAM ,
@@ -287,6 +287,19 @@ such as error messages.
 Therefore, a message accompanied by a particular
 .Fa sc_euid
 value should not be trusted as being from that user.
+.It Dv LOCAL_CREDS_PERSISTENT
+This option is similar to
+.Dv LOCAL_CREDS ,
+except that socket credentials are passed on every read from a
+.Dv SOCK_STREAM
+or
+.Dv SOCK_SEQPACKET
+socket, instead of just the first read.
+The
+.Dv LOCAL_CREDS
+and
+.Dv LOCAL_CREDS_PERSISTENT
+options are mutually exclusive.
 .It Dv LOCAL_CONNWAIT
 Used with
 .Dv SOCK_STREAM

Modified: head/sys/kern/uipc_usrreq.c
==
--- head/sys/kern/uipc_usrreq.c Tue Nov  3 01:10:27 2020(r367286)
+++ head/sys/kern/uipc_usrreq.c Tue Nov  3 01:17:45 2020(r367287)
@@ -1040,7 +1040,7 @@ uipc_send(struct socket *so, int flags, struct mbuf *m
break;
}
 
-   if (unp2->unp_flags & UNP_WANTCRED)
+   if (unp2->unp_flags & UNP_WANTCRED_MASK)
control = unp_addsockcred(td, control);
if (unp->unp_addr != NULL)
from = (struct sockaddr *)unp->unp_addr;
@@ -1094,12 +1094,13 @@ uipc_send(struct socket *so, int flags, struct mbuf *m
break;
}
SOCKBUF_LOCK(>so_rcv);
-   if (unp2->unp_flags & UNP_WANTCRED) {
+   if (unp2->unp_flags & UNP_WANTCRED_MASK) {
/*
-* Credentials are passed only once on SOCK_STREAM
-* and SOCK_SEQPACKET.
+* Credentials are passed only once on SOCK_STREAM and
+* SOCK_SEQPACKET (LOCAL_CREDS => WANTCRED_ONESHOT), or
+* forever (LOCAL_CREDS_PERSISTENT => WANTCRED_ALWAYS).
 */
-   unp2->unp_flags &= ~UNP_WANTCRED;
+   unp2->unp_flags &= ~UNP_WANTCRED_ONESHOT;
control = unp_addsockcred(td, control);
}
 
@@ -1405,10 +1406,16 @@ uipc_ctloutput(struct socket *so, struct sockopt *sopt
 
case LOCAL_CREDS:
/* Unlocked read. */
-   optval = unp->unp_flags & UNP_WANTCRED ? 1 : 0;
+   optval = unp->unp_flags & UNP_WANTCRED_ONESHOT ? 1 : 0;
error = sooptcopyout(sopt, , sizeof(optval));
break;
 
+   case LOCAL_CREDS_PERSISTENT:
+   /* Unlocked read. */
+   optval = unp->unp_flags & UNP_WANTCRED_ALWAYS ? 1 : 0;
+   error = sooptcopyout(sopt, , sizeof(optval));
+   break;
+
case LOCAL_CONNWAIT:
/* Unlocked read. */
optval = unp->unp_flags & UNP_CONNWAIT ? 1 : 0;
@@ -1424,28 +1431,38 @@ uipc_ctloutput(struct socket *so, struct sockopt *sopt
case SOPT_SET:
switch (sopt->sopt_name) {
case LOCAL_CREDS:
+   case LOCAL_CREDS_PERSISTENT:
case LOCAL_CONNWAIT:
error = sooptcopyin(sopt, , sizeof(optval),
sizeof(optval));
if (error)
break;
 
-#defineOPTSET(bit) do {
\
+#defineOPTSET(bit, exclusive) do { 
\
UNP_PCB_LOCK(unp);  \
-   if (optval) \
-   

svn commit: r367286 - head/sys/compat/linux

2020-11-02 Thread Conrad Meyer
Author: cem
Date: Tue Nov  3 01:10:27 2020
New Revision: 367286
URL: https://svnweb.freebsd.org/changeset/base/367286

Log:
  linux(4): style: Eliminate dead 'break' after 'return'
  
  No functional change.

Modified:
  head/sys/compat/linux/linux_misc.c

Modified: head/sys/compat/linux/linux_misc.c
==
--- head/sys/compat/linux/linux_misc.c  Mon Nov  2 21:47:34 2020
(r367285)
+++ head/sys/compat/linux/linux_misc.c  Tue Nov  3 01:10:27 2020
(r367286)
@@ -1955,7 +1955,6 @@ linux_prctl(struct thread *td, struct linux_prctl_args
return (copyout(_signal,
(void *)(register_t)args->arg2,
sizeof(pdeath_signal)));
-   break;
case LINUX_PR_SET_DUMPABLE:
linux_msg(td, "unsupported prctl PR_SET_DUMPABLE");
error = EINVAL;
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


Re: svn commit: r367280 - head/lib/libc/gen

2020-11-02 Thread Stefan Esser

Am 02.11.20 um 23:10 schrieb Konstantin Belousov:

On Mon, Nov 02, 2020 at 10:49:07PM +0100, Emmanuel Vadot wrote:

  I think that the first question we want to ask is : Do we want to
support LOCALBASE being different than /usr/local
  I honestly don't see any advantages of making it !=/usr/local/ and
before we start putting a lot of new/useless(for I guess 99% of our
user base) in the tree we should here why people are using /usr/pkg or
whatever weird location.
  If they have some good argument, then we should proceed further.


I would be delighted to be able to install _and use_ two independent
set of packages from the same base system install.  Without recursing
to jails, X forwarding, etc.


I understand the use case, and I agree this may be appropriate for
a development system.

But on a production system I'd never want to have a non-constant and
not generally applied LOCALBASE, at least not on a system that gives
a CLI to unprivileged users. Those could build their own copy of the
LOCALBASE tree (e.g. sym-linking all sub-trees that are to be kept
unmodified, replacing config files that policies that restrict the
user).

And if LOCALBASE is not compiled into binaries but somehow obtained
at run-time, there are a number of attacks I can imagine (e.g. by
LD_PRELOAD replace the sysctl() call in libc by your own version).


In fact I would like to use /usr/local and e.g /usr/local-i386 on amd64
machine.  I am fine with me building both of them in my instance of
poudriere.


This is a use-case for architecture dependent path definitions (which
I have used some 30 years ago on HP-UX which supported 68k and HP-PA
on a single file system that way). Such a feature has been discussed
in FreeBSD multiple times over the decades ...


But indeed I am not sure if this worth the effort of many people, for many
hours.  If it puts too high burden on everybody, then it is not a good
feature.  Otherwise, it is very convenient in some situations.


Up to now, not much effort has been spent on this issue. I have defined
_PATH_LOCALBASE in paths.h to make it available for the calendar program
to let it lookup port provided data files (and moving those to a port
will simplify the maintenance of these calendar files).

With _PATH_LOCALBASE available, it was sensible to replace literal uses
of /usr/local in the tree with references to this symbolic name (and
thus to easily build a base system for a different LOCALBASE value).

The addition of the user.localbase sysctl tool less than 20 lines of
code in 2 files (sys/kern_mib.c and lib/libc/gen/sysctl.c). This value
can now be used to make /etc/defaults/rc.conf adapt to a changed
LOCALBASE (just a few lines changed).

All in all less than 50 lines of code have been affected, but this
makes it much easier to build a system for a different (fixed)
LOCALBASE.

The getlocalbase() function proposed by Scott Long will allow to use
a user-configured LOCALBASE in programs that use this function instead
of a compiled in path. Another 20 lines of code, not complicated at all.

But the security implications of the use of a dynamic LOCALBASE can be
significant. And thus I do not think that we should put this function
into programs without a prior security assessment. And it would be a
major effort, which IMHO is not justified for most programs.

But this is not an argument against the steps that have been taken up
to now, with little effort and no weakening of security.

Best regards, STefan


OpenPGP_signature
Description: OpenPGP digital signature


Re: svn commit: r367280 - head/lib/libc/gen

2020-11-02 Thread Emmanuel Vadot
On Tue, 3 Nov 2020 00:32:14 +0200
Konstantin Belousov  wrote:

> On Mon, Nov 02, 2020 at 11:22:15PM +0100, Emmanuel Vadot wrote:
> > On Tue, 3 Nov 2020 00:10:39 +0200
> > Konstantin Belousov  wrote:
> > 
> > > On Mon, Nov 02, 2020 at 10:49:07PM +0100, Emmanuel Vadot wrote:
> > > >  I think that the first question we want to ask is : Do we want to
> > > > support LOCALBASE being different than /usr/local
> > > >  I honestly don't see any advantages of making it !=/usr/local/ and
> > > > before we start putting a lot of new/useless(for I guess 99% of our
> > > > user base) in the tree we should here why people are using /usr/pkg or
> > > > whatever weird location.
> > > >  If they have some good argument, then we should proceed further.
> > > 
> > > I would be delighted to be able to install _and use_ two independent
> > > set of packages from the same base system install.  Without recursing
> > > to jails, X forwarding, etc.
> > > 
> > > In fact I would like to use /usr/local and e.g /usr/local-i386 on amd64
> > > machine.  I am fine with me building both of them in my instance of
> > > poudriere.
> > > 
> > > But indeed I am not sure if this worth the effort of many people, for many
> > > hours.  If it puts too high burden on everybody, then it is not a good
> > > feature.  Otherwise, it is very convenient in some situations.
> > 
> >  I understand this situation but I think that the best way for you do
> > do that is to use pkg install -r /path/to/my/i386/packages
> > 
> >  Since you will need to tweak you PATH variable to start applications
> > installed in /usr/local-i386 anyway it's not too much to tweak that to
> > the pkg path for your i386 repo.
> > 
> >  The "downside" of using this method is that you will have
> > a /usr/local/ under the /path/to/my/i386/packages.
> >  The "upside" of using this method is that you would be able to use the
> > same i386 packages on a native i386 install and they would install
> > in /usr/local/ (so no tweaking here).
> If I can already use them from non-/usr/local prefix, then it is great
> news (for me).  But I have a reason to doubt.

 If you pkg -r packages you can use a lot of them.

> For instance, a lot of applications are configured at build time to look
> for /usr/local.  Like, gcc with /usr/local/lib/gcc/, and binutils,
> which are actually one of the main use case for me.  So I believe that
> pkg install -r requires chroot/jail for the result to work.

 Yes there is still some cases like that, or packages having
post-install script that don't handle -r.
 We've been working on that with bapt@ for a few months now and still
do. The main motivation of rewriting everything in lua is to be able to
do that but there is still a lot do to.
 Never the less we would appriciate some reports of people using
packages installed with -r.

-- 
Emmanuel Vadot 
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


Re: svn commit: r367280 - head/lib/libc/gen

2020-11-02 Thread Emmanuel Vadot
On Mon, 2 Nov 2020 23:24:34 +0100
Stefan Esser  wrote:

> Am 02.11.20 um 22:49 schrieb Emmanuel Vadot:
> > On Mon, 2 Nov 2020 22:41:38 +0100
> > Stefan Esser  wrote:
> > 
> >> Am 02.11.20 um 20:20 schrieb Oliver Pinter:> On Monday, November 2,
> >> 2020, Stefan Eßer  >>> > wrote:
> >>>
> >>>  Author: se
> >>>  Date: Mon Nov  2 18:48:06 2020
> >>>  New Revision: 367280
> >>>  URL: https://svnweb.freebsd.org/changeset/base/367280
> >>>  
> >>>
> >>>  Log:
> >>>     Re-arrange some of the code to separate writable user tree
> >>>  variables from
> >>>     R/O variables.
> >>>
> >>>     While here fix some nearby style. No functional change intended.
> >>>
> >>>     MFC after:    1 month
> >>>
> >>>
> >>> Is there any phabricator reference for this / these commit(s) + reviewer
> >>> lists?
> >>
> >> The previous commit that has been refined in this one has been
> >> discussed in D27009.
> >>
> >> I had added the new R/W sysctl variable to a switch statement that
> >> contained one R/O string value, and excluded the OID from causing
> >> an error return when a new value had been passed.
> >>
> >> This was functionally OK, but I have decided to move handling of
> >> the new writable variable to before the check for a write attempt
> >> and thus need to test specifically for its OID.
> >>
> >> This sysctl variable is referenced in Scott Longs proposed
> >> getlocalbase() function (D27022), but also in the change to make
> >> it define defaults paths in /etc/defaults/rc.conf (D27014).
> >>
> >> I do not support to make LOCALBASE dynamic for a broad range of
> >> programs, since this could lead to severe security issues (e.g.
> >> when a program is restricted by policy settings LOCALBASE/etc and
> >> an user-defined LOCALBASE could be used to circumvent them.
> >>
> >> There are already programs that respect a LOCALBASE environment
> >> variable, e.g. the pkg program, to allow it to e.g. operate with
> >> a DESTDIR prefix other than "/". This is a program that could
> >> instead use getlocalbase(), IMHO.
> >>
> >> But for security reasons all files that determine policies and
> >> exist in LOCALBASE since they are not distributed as part of the
> >> base system, should be located in a secure way, and that is by
> >> referring to a compiled in trusted path, IMHO.
> >>
> >> Even if the sysctl variable "user.localbase" can only be written to
> >> by root, the use of getlocalbase() provided by a shared library could
> >> allow to perform a LD_PRELOAD attack (provide a getlocalbase() that
> >> leadsto a user provided policy file instead of the admin controlled
> >> one).
> >>
> >> Regards, STefan
> > 
> >   I think that the first question we want to ask is : Do we want to
> > support LOCALBASE being different than /usr/local
> 
> The big majority of users will keep the default value, and I do not
> see a good reason for a change, except if there is a large installed
> base that traditionally uses another prefix (I have seen /vol/local
> and /opt, but also OS and architecture-specific prefixes, for example).

 I'd still like to see some arguments for such installs.

> >   I honestly don't see any advantages of making it !=/usr/local/ and
> > before we start putting a lot of new/useless(for I guess 99% of our
> > user base) in the tree we should here why people are using /usr/pkg or
> > whatever weird location.
> 
> No, why should we [assess] (assuming that word is to be implied in
> your sentence) why people want to be able to easily use a different
> prefix? That would be a waste of time, IMHO.
> 
> I know that there are legitimate reasons to want a different prefix,
> and we had requests to make it easier to support it.

 What are thoses ?

> We have literal uses of /usr/local in a lot of files in the FreeBSD
> base system (more than 1700) and this is not going to change.
> 
> But it was easy to replace a number of such literal pathes in base
> system binaries, and we can make it easier for those that need a
> different prefix to get it consistently used.
> 
> >   If they have some good argument, then we should proceed further.
> 
> You do not have to participate in this effort 

 I do have to participate, it's a common project.
 Also since I also participate in pkg(8) and in ports/Mk lua/blah stuff
there might be some stuff to do there so yes I need to participate.
 And since you never really started a conversation on a ml (that I know
of) my only mean to start this participation is answering a commit
email.

> - there are so many
> other areas to work on (and I know you are very active in one).

 Only one ? Damn, I should work more then.

> But please do not ask those that have started to reduce the use of
> literal /usr/local in the base system to justify this work.

 Seriously ? I have every right to ask you to justify this when it was
not talked about in a public forum.

> If you are happy with 

Re: svn commit: r367280 - head/lib/libc/gen

2020-11-02 Thread Konstantin Belousov
On Mon, Nov 02, 2020 at 11:22:15PM +0100, Emmanuel Vadot wrote:
> On Tue, 3 Nov 2020 00:10:39 +0200
> Konstantin Belousov  wrote:
> 
> > On Mon, Nov 02, 2020 at 10:49:07PM +0100, Emmanuel Vadot wrote:
> > >  I think that the first question we want to ask is : Do we want to
> > > support LOCALBASE being different than /usr/local
> > >  I honestly don't see any advantages of making it !=/usr/local/ and
> > > before we start putting a lot of new/useless(for I guess 99% of our
> > > user base) in the tree we should here why people are using /usr/pkg or
> > > whatever weird location.
> > >  If they have some good argument, then we should proceed further.
> > 
> > I would be delighted to be able to install _and use_ two independent
> > set of packages from the same base system install.  Without recursing
> > to jails, X forwarding, etc.
> > 
> > In fact I would like to use /usr/local and e.g /usr/local-i386 on amd64
> > machine.  I am fine with me building both of them in my instance of
> > poudriere.
> > 
> > But indeed I am not sure if this worth the effort of many people, for many
> > hours.  If it puts too high burden on everybody, then it is not a good
> > feature.  Otherwise, it is very convenient in some situations.
> 
>  I understand this situation but I think that the best way for you do
> do that is to use pkg install -r /path/to/my/i386/packages
> 
>  Since you will need to tweak you PATH variable to start applications
> installed in /usr/local-i386 anyway it's not too much to tweak that to
> the pkg path for your i386 repo.
> 
>  The "downside" of using this method is that you will have
> a /usr/local/ under the /path/to/my/i386/packages.
>  The "upside" of using this method is that you would be able to use the
> same i386 packages on a native i386 install and they would install
> in /usr/local/ (so no tweaking here).
If I can already use them from non-/usr/local prefix, then it is great
news (for me).  But I have a reason to doubt.

For instance, a lot of applications are configured at build time to look
for /usr/local.  Like, gcc with /usr/local/lib/gcc/, and binutils,
which are actually one of the main use case for me.  So I believe that
pkg install -r requires chroot/jail for the result to work.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


Re: svn commit: r367280 - head/lib/libc/gen

2020-11-02 Thread Stefan Esser

Am 02.11.20 um 22:49 schrieb Emmanuel Vadot:

On Mon, 2 Nov 2020 22:41:38 +0100
Stefan Esser  wrote:


Am 02.11.20 um 20:20 schrieb Oliver Pinter:> On Monday, November 2,
2020, Stefan Eßer 
> wrote:

 Author: se
 Date: Mon Nov  2 18:48:06 2020
 New Revision: 367280
 URL: https://svnweb.freebsd.org/changeset/base/367280
 

 Log:
    Re-arrange some of the code to separate writable user tree
 variables from
    R/O variables.

    While here fix some nearby style. No functional change intended.

    MFC after:    1 month


Is there any phabricator reference for this / these commit(s) + reviewer
lists?


The previous commit that has been refined in this one has been
discussed in D27009.

I had added the new R/W sysctl variable to a switch statement that
contained one R/O string value, and excluded the OID from causing
an error return when a new value had been passed.

This was functionally OK, but I have decided to move handling of
the new writable variable to before the check for a write attempt
and thus need to test specifically for its OID.

This sysctl variable is referenced in Scott Longs proposed
getlocalbase() function (D27022), but also in the change to make
it define defaults paths in /etc/defaults/rc.conf (D27014).

I do not support to make LOCALBASE dynamic for a broad range of
programs, since this could lead to severe security issues (e.g.
when a program is restricted by policy settings LOCALBASE/etc and
an user-defined LOCALBASE could be used to circumvent them.

There are already programs that respect a LOCALBASE environment
variable, e.g. the pkg program, to allow it to e.g. operate with
a DESTDIR prefix other than "/". This is a program that could
instead use getlocalbase(), IMHO.

But for security reasons all files that determine policies and
exist in LOCALBASE since they are not distributed as part of the
base system, should be located in a secure way, and that is by
referring to a compiled in trusted path, IMHO.

Even if the sysctl variable "user.localbase" can only be written to
by root, the use of getlocalbase() provided by a shared library could
allow to perform a LD_PRELOAD attack (provide a getlocalbase() that
leadsto a user provided policy file instead of the admin controlled
one).

Regards, STefan


  I think that the first question we want to ask is : Do we want to
support LOCALBASE being different than /usr/local


The big majority of users will keep the default value, and I do not
see a good reason for a change, except if there is a large installed
base that traditionally uses another prefix (I have seen /vol/local
and /opt, but also OS and architecture-specific prefixes, for example).


  I honestly don't see any advantages of making it !=/usr/local/ and
before we start putting a lot of new/useless(for I guess 99% of our
user base) in the tree we should here why people are using /usr/pkg or
whatever weird location.


No, why should we [assess] (assuming that word is to be implied in
your sentence) why people want to be able to easily use a different
prefix? That would be a waste of time, IMHO.

I know that there are legitimate reasons to want a different prefix,
and we had requests to make it easier to support it.

We have literal uses of /usr/local in a lot of files in the FreeBSD
base system (more than 1700) and this is not going to change.

But it was easy to replace a number of such literal pathes in base
system binaries, and we can make it easier for those that need a
different prefix to get it consistently used.


  If they have some good argument, then we should proceed further.


You do not have to participate in this effort - there are so many
other areas to work on (and I know you are very active in one).

But please do not ask those that have started to reduce the use of
literal /usr/local in the base system to justify this work.

If you are happy with /usr/local, then you are not affected at all.
And if you need to configure your system to use a different prefix,
you are welcome to let us know which steps are still causing much
effort and should be worked on to make it easier ...

Do you have any reason to be against removal of literal /usr/local
from the base system in favor of using a symbolic name for it?

Regards, STefan
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


Re: svn commit: r367280 - head/lib/libc/gen

2020-11-02 Thread Emmanuel Vadot
On Tue, 3 Nov 2020 00:10:39 +0200
Konstantin Belousov  wrote:

> On Mon, Nov 02, 2020 at 10:49:07PM +0100, Emmanuel Vadot wrote:
> >  I think that the first question we want to ask is : Do we want to
> > support LOCALBASE being different than /usr/local
> >  I honestly don't see any advantages of making it !=/usr/local/ and
> > before we start putting a lot of new/useless(for I guess 99% of our
> > user base) in the tree we should here why people are using /usr/pkg or
> > whatever weird location.
> >  If they have some good argument, then we should proceed further.
> 
> I would be delighted to be able to install _and use_ two independent
> set of packages from the same base system install.  Without recursing
> to jails, X forwarding, etc.
> 
> In fact I would like to use /usr/local and e.g /usr/local-i386 on amd64
> machine.  I am fine with me building both of them in my instance of
> poudriere.
> 
> But indeed I am not sure if this worth the effort of many people, for many
> hours.  If it puts too high burden on everybody, then it is not a good
> feature.  Otherwise, it is very convenient in some situations.

 I understand this situation but I think that the best way for you do
do that is to use pkg install -r /path/to/my/i386/packages

 Since you will need to tweak you PATH variable to start applications
installed in /usr/local-i386 anyway it's not too much to tweak that to
the pkg path for your i386 repo.

 The "downside" of using this method is that you will have
a /usr/local/ under the /path/to/my/i386/packages.
 The "upside" of using this method is that you would be able to use the
same i386 packages on a native i386 install and they would install
in /usr/local/ (so no tweaking here).

-- 
Emmanuel Vadot 
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


Re: svn commit: r367280 - head/lib/libc/gen

2020-11-02 Thread Konstantin Belousov
On Mon, Nov 02, 2020 at 10:49:07PM +0100, Emmanuel Vadot wrote:
>  I think that the first question we want to ask is : Do we want to
> support LOCALBASE being different than /usr/local
>  I honestly don't see any advantages of making it !=/usr/local/ and
> before we start putting a lot of new/useless(for I guess 99% of our
> user base) in the tree we should here why people are using /usr/pkg or
> whatever weird location.
>  If they have some good argument, then we should proceed further.

I would be delighted to be able to install _and use_ two independent
set of packages from the same base system install.  Without recursing
to jails, X forwarding, etc.

In fact I would like to use /usr/local and e.g /usr/local-i386 on amd64
machine.  I am fine with me building both of them in my instance of
poudriere.

But indeed I am not sure if this worth the effort of many people, for many
hours.  If it puts too high burden on everybody, then it is not a good
feature.  Otherwise, it is very convenient in some situations.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


Re: svn commit: r367280 - head/lib/libc/gen

2020-11-02 Thread Emmanuel Vadot
On Mon, 2 Nov 2020 22:41:38 +0100
Stefan Esser  wrote:

> Am 02.11.20 um 20:20 schrieb Oliver Pinter:> On Monday, November 2, 
> 2020, Stefan Eßer  > > wrote:
> > 
> > Author: se
> > Date: Mon Nov  2 18:48:06 2020
> > New Revision: 367280
> > URL: https://svnweb.freebsd.org/changeset/base/367280
> > 
> > 
> > Log:
> >    Re-arrange some of the code to separate writable user tree
> > variables from
> >    R/O variables.
> > 
> >    While here fix some nearby style. No functional change intended.
> > 
> >    MFC after:    1 month
> > 
> > 
> > Is there any phabricator reference for this / these commit(s) + reviewer 
> > lists?
> 
> The previous commit that has been refined in this one has been
> discussed in D27009.
> 
> I had added the new R/W sysctl variable to a switch statement that
> contained one R/O string value, and excluded the OID from causing
> an error return when a new value had been passed.
> 
> This was functionally OK, but I have decided to move handling of
> the new writable variable to before the check for a write attempt
> and thus need to test specifically for its OID.
> 
> This sysctl variable is referenced in Scott Longs proposed
> getlocalbase() function (D27022), but also in the change to make
> it define defaults paths in /etc/defaults/rc.conf (D27014).
> 
> I do not support to make LOCALBASE dynamic for a broad range of
> programs, since this could lead to severe security issues (e.g.
> when a program is restricted by policy settings LOCALBASE/etc and
> an user-defined LOCALBASE could be used to circumvent them.
> 
> There are already programs that respect a LOCALBASE environment
> variable, e.g. the pkg program, to allow it to e.g. operate with
> a DESTDIR prefix other than "/". This is a program that could
> instead use getlocalbase(), IMHO.
> 
> But for security reasons all files that determine policies and
> exist in LOCALBASE since they are not distributed as part of the
> base system, should be located in a secure way, and that is by
> referring to a compiled in trusted path, IMHO.
> 
> Even if the sysctl variable "user.localbase" can only be written to
> by root, the use of getlocalbase() provided by a shared library could
> allow to perform a LD_PRELOAD attack (provide a getlocalbase() that
> leadsto a user provided policy file instead of the admin controlled
> one).
> 
> Regards, STefan

 I think that the first question we want to ask is : Do we want to
support LOCALBASE being different than /usr/local
 I honestly don't see any advantages of making it !=/usr/local/ and
before we start putting a lot of new/useless(for I guess 99% of our
user base) in the tree we should here why people are using /usr/pkg or
whatever weird location.
 If they have some good argument, then we should proceed further.

-- 
Emmanuel Vadot 
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


svn commit: r367285 - head/sbin/ifconfig

2020-11-02 Thread Konstantin Belousov
Author: kib
Date: Mon Nov  2 21:47:34 2020
New Revision: 367285
URL: https://svnweb.freebsd.org/changeset/base/367285

Log:
  ifconfig: properly detect invalid mediaopt keywords.
  
  When invalid keyword is specified, ifconfig(8) is silent about it,
  instead random request is sent to the driver.
  
  Before the patch:
  root@r-freeb43:~ # ifconfig mce0 mediaopt -txpause,-rxpause
  ifconfig: SIOCSIFMEDIA (media): Device not configured
  
  After:
  root@r-freeb43:~ # ifconfig mce0 mediaopt -txpause,-rxpause
  ifconfig: unknown option: -txpause
  
  Reviewed by:  hselasky, kp
  Sponsored by: Mellanox Technologies / NVidia Networking
  MFC after:1 week
  Differential revision:https://reviews.freebsd.org/D27060

Modified:
  head/sbin/ifconfig/ifmedia.c

Modified: head/sbin/ifconfig/ifmedia.c
==
--- head/sbin/ifconfig/ifmedia.cMon Nov  2 21:10:49 2020
(r367284)
+++ head/sbin/ifconfig/ifmedia.cMon Nov  2 21:47:34 2020
(r367285)
@@ -566,7 +566,7 @@ get_media_options(int type, const char *val)
struct ifmedia_description *desc;
struct ifmedia_type_to_subtype *ttos;
char *optlist, *optptr;
-   int option = 0, i, rval = 0;
+   int option, i, rval = 0;
 
/* We muck with the string, so copy it. */
optlist = strdup(val);
@@ -587,12 +587,13 @@ get_media_options(int type, const char *val)
 */
optptr = optlist;
for (; (optptr = strtok(optptr, ",")) != NULL; optptr = NULL) {
+   option = -1;
for (i = 0; ttos->options[i].desc != NULL; i++) {
option = lookup_media_word(ttos->options[i].desc, 
optptr);
if (option != -1)
break;
}
-   if (option == 0)
+   if (option == -1)
errx(1, "unknown option: %s", optptr);
rval |= option;
}
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


Re: svn commit: r367280 - head/lib/libc/gen

2020-11-02 Thread Stefan Esser
Am 02.11.20 um 20:20 schrieb Oliver Pinter:> On Monday, November 2, 
2020, Stefan Eßer 
> wrote:

Author: se
Date: Mon Nov  2 18:48:06 2020
New Revision: 367280
URL: https://svnweb.freebsd.org/changeset/base/367280


Log:
   Re-arrange some of the code to separate writable user tree
variables from
   R/O variables.

   While here fix some nearby style. No functional change intended.

   MFC after:    1 month


Is there any phabricator reference for this / these commit(s) + reviewer 
lists?


The previous commit that has been refined in this one has been
discussed in D27009.

I had added the new R/W sysctl variable to a switch statement that
contained one R/O string value, and excluded the OID from causing
an error return when a new value had been passed.

This was functionally OK, but I have decided to move handling of
the new writable variable to before the check for a write attempt
and thus need to test specifically for its OID.

This sysctl variable is referenced in Scott Longs proposed
getlocalbase() function (D27022), but also in the change to make
it define defaults paths in /etc/defaults/rc.conf (D27014).

I do not support to make LOCALBASE dynamic for a broad range of
programs, since this could lead to severe security issues (e.g.
when a program is restricted by policy settings LOCALBASE/etc and
an user-defined LOCALBASE could be used to circumvent them.

There are already programs that respect a LOCALBASE environment
variable, e.g. the pkg program, to allow it to e.g. operate with
a DESTDIR prefix other than "/". This is a program that could
instead use getlocalbase(), IMHO.

But for security reasons all files that determine policies and
exist in LOCALBASE since they are not distributed as part of the
base system, should be located in a secure way, and that is by
referring to a compiled in trusted path, IMHO.

Even if the sysctl variable "user.localbase" can only be written to
by root, the use of getlocalbase() provided by a shared library could
allow to perform a LD_PRELOAD attack (provide a getlocalbase() that
leadsto a user provided policy file instead of the admin controlled
one).

Regards, STefan


OpenPGP_signature
Description: OpenPGP digital signature


svn commit: r367284 - in head/release: . tools

2020-11-02 Thread Emmanuel Vadot
Author: manu
Date: Mon Nov  2 21:10:49 2020
New Revision: 367284
URL: https://svnweb.freebsd.org/changeset/base/367284

Log:
  release: Add an image for CI
  
  A lot of projects CI can't do FreeBSD tests currently.
  The main reason is that the project CI infrastructure is runned on Linux
  and that our images aren't modifiable from a Linux hosts.
  Add a basic image specific for this case (called BASIC-CI for a lack of a
  better name).
  The image have no package pre-installed.
  It only have a few modification to have dhcp client runned on the default
  interface and sshd started with option to be able to log on without a password
  as root.
  
  Sponsored by: The FreeBSD Foundation
  
  Reviewed by:  re (gjb@)
  Differential Revision:https://reviews.freebsd.org/D25598

Added:
  head/release/tools/basic-ci.conf   (contents, props changed)
Modified:
  head/release/Makefile.vm

Modified: head/release/Makefile.vm
==
--- head/release/Makefile.vmMon Nov  2 20:00:50 2020(r367283)
+++ head/release/Makefile.vmMon Nov  2 21:10:49 2020(r367284)
@@ -16,13 +16,17 @@ VMDK_DESC=  VMWare, VirtualBox disk image
 QCOW2_DESC=Qemu, KVM disk image
 RAW_DESC=  Unformatted raw disk image
 
-CLOUDWARE?=EC2 \
+CLOUDWARE?=BASIC-CI \
+   EC2 \
GCE \
VAGRANT-VIRTUALBOX \
VAGRANT-VMWARE
 AZURE_FORMAT=  vhdf
 AZURE_DESC=Microsoft Azure platform image
 AZURE_DISK=${OSRELEASE}.${AZURE_FORMAT}
+BASIC-CI_FORMAT=   raw
+BASIC-CI_DESC= Image for CI
+BASIC-CI_DISK= ${OSRELEASE}.${BASIC-CI_FORMAT}
 EC2_FORMAT=raw
 EC2_DESC=  Amazon EC2 image
 EC2_DISK=  ${OSRELEASE}.${EC2_FORMAT}

Added: head/release/tools/basic-ci.conf
==
--- /dev/null   00:00:00 1970   (empty, because file is newly added)
+++ head/release/tools/basic-ci.confMon Nov  2 21:10:49 2020
(r367284)
@@ -0,0 +1,35 @@
+#!/bin/sh
+#
+# $FreeBSD$
+#
+
+# Should be enough for base image, image can be resized in needed
+export VMSIZE=5G
+
+# Set to a list of third-party software to enable in rc.conf(5).
+export VM_RC_LIST="sshd growfs"
+
+vm_extra_pre_umount() {
+   cat << EOF >> ${DESTDIR}/etc/rc.conf
+dumpdev="AUTO"
+ifconfig_DEFAULT="DHCP"
+sshd_enable="YES"
+EOF
+
+   cat << EOF >> ${DESTDIR}/boot/loader.conf
+autoboot_delay="-1"
+beastie_disable="YES"
+loader_logo="none"
+console="comconsole,vidconsole"
+EOF
+cat <> ${DESTDIR}/etc/ssh/sshd_config
+PermitRootLogin yes
+PasswordAuthentication yes
+PermitEmptyPasswords yes
+UsePAM no
+EOF
+
+   touch ${DESTDIR}/firstboot
+
+   return 0
+}
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


svn commit: r367282 - in head/sys/arm64: arm64 include

2020-11-02 Thread Ruslan Bukin
Author: br
Date: Mon Nov  2 19:56:15 2020
New Revision: 367282
URL: https://svnweb.freebsd.org/changeset/base/367282

Log:
  Add routines for ARM System MMU (SMMU) pmap management.
  
  Reviewed by:  markj
  Discussed with:   kib
  Sponsored by: DARPA, Innovate UK
  Differential Revision:https://reviews.freebsd.org/D26877

Modified:
  head/sys/arm64/arm64/pmap.c
  head/sys/arm64/include/pmap.h

Modified: head/sys/arm64/arm64/pmap.c
==
--- head/sys/arm64/arm64/pmap.c Mon Nov  2 19:20:06 2020(r367281)
+++ head/sys/arm64/arm64/pmap.c Mon Nov  2 19:56:15 2020(r367282)
@@ -3605,6 +3605,184 @@ restart:
 }
 
 /*
+ * Add a single SMMU entry. This function does not sleep.
+ */
+int
+pmap_senter(pmap_t pmap, vm_offset_t va, vm_paddr_t pa,
+vm_prot_t prot, u_int flags)
+{
+   pd_entry_t *pde;
+   pt_entry_t new_l3, orig_l3;
+   pt_entry_t *l3;
+   vm_page_t mpte;
+   int lvl;
+   int rv;
+
+   PMAP_ASSERT_STAGE1(pmap);
+   KASSERT(va < VM_MAXUSER_ADDRESS, ("wrong address space"));
+
+   va = trunc_page(va);
+   new_l3 = (pt_entry_t)(pa | ATTR_DEFAULT |
+   ATTR_S1_IDX(VM_MEMATTR_DEVICE) | L3_PAGE);
+   if ((prot & VM_PROT_WRITE) == 0)
+   new_l3 |= ATTR_S1_AP(ATTR_S1_AP_RO);
+   new_l3 |= ATTR_S1_XN; /* Execute never. */
+   new_l3 |= ATTR_S1_AP(ATTR_S1_AP_USER);
+   new_l3 |= ATTR_S1_nG; /* Non global. */
+
+   CTR2(KTR_PMAP, "pmap_senter: %.16lx -> %.16lx", va, pa);
+
+   PMAP_LOCK(pmap);
+
+   /*
+* In the case that a page table page is not
+* resident, we are creating it here.
+*/
+retry:
+   pde = pmap_pde(pmap, va, );
+   if (pde != NULL && lvl == 2) {
+   l3 = pmap_l2_to_l3(pde, va);
+   } else {
+   mpte = _pmap_alloc_l3(pmap, pmap_l2_pindex(va), NULL);
+   if (mpte == NULL) {
+   CTR0(KTR_PMAP, "pmap_enter: mpte == NULL");
+   rv = KERN_RESOURCE_SHORTAGE;
+   goto out;
+   }
+   goto retry;
+   }
+
+   orig_l3 = pmap_load(l3);
+   KASSERT(!pmap_l3_valid(orig_l3), ("l3 is valid"));
+
+   /* New mapping */
+   pmap_store(l3, new_l3);
+   pmap_resident_count_inc(pmap, 1);
+   dsb(ishst);
+
+   rv = KERN_SUCCESS;
+out:
+   PMAP_UNLOCK(pmap);
+
+   return (rv);
+}
+
+/*
+ * Remove a single SMMU entry.
+ */
+int
+pmap_sremove(pmap_t pmap, vm_offset_t va)
+{
+   pt_entry_t *pte;
+   int lvl;
+   int rc;
+
+   PMAP_LOCK(pmap);
+
+   pte = pmap_pte(pmap, va, );
+   KASSERT(lvl == 3,
+   ("Invalid SMMU pagetable level: %d != 3", lvl));
+
+   if (pte != NULL) {
+   pmap_resident_count_dec(pmap, 1);
+   pmap_clear(pte);
+   rc = KERN_SUCCESS;
+   } else
+   rc = KERN_FAILURE;
+
+   PMAP_UNLOCK(pmap);
+
+   return (rc);
+}
+
+/*
+ * Remove all the allocated L1, L2 pages from SMMU pmap.
+ * All the L3 entires must be cleared in advance, otherwise
+ * this function panics.
+ */
+void
+pmap_sremove_pages(pmap_t pmap)
+{
+   pd_entry_t l0e, *l1, l1e, *l2, l2e;
+   pt_entry_t *l3, l3e;
+   vm_page_t m, m0, m1;
+   vm_offset_t sva;
+   vm_paddr_t pa;
+   vm_paddr_t pa0;
+   vm_paddr_t pa1;
+   int i, j, k, l;
+
+   PMAP_LOCK(pmap);
+
+   for (sva = VM_MINUSER_ADDRESS, i = pmap_l0_index(sva);
+   (i < Ln_ENTRIES && sva < VM_MAXUSER_ADDRESS); i++) {
+   l0e = pmap->pm_l0[i];
+   if ((l0e & ATTR_DESCR_VALID) == 0) {
+   sva += L0_SIZE;
+   continue;
+   }
+   pa0 = l0e & ~ATTR_MASK;
+   m0 = PHYS_TO_VM_PAGE(pa0);
+   l1 = (pd_entry_t *)PHYS_TO_DMAP(pa0);
+
+   for (j = pmap_l1_index(sva); j < Ln_ENTRIES; j++) {
+   l1e = l1[j];
+   if ((l1e & ATTR_DESCR_VALID) == 0) {
+   sva += L1_SIZE;
+   continue;
+   }
+   if ((l1e & ATTR_DESCR_MASK) == L1_BLOCK) {
+   sva += L1_SIZE;
+   continue;
+   }
+   pa1 = l1e & ~ATTR_MASK;
+   m1 = PHYS_TO_VM_PAGE(pa1);
+   l2 = (pd_entry_t *)PHYS_TO_DMAP(pa1);
+
+   for (k = pmap_l2_index(sva); k < Ln_ENTRIES; k++) {
+   l2e = l2[k];
+   if ((l2e & ATTR_DESCR_VALID) == 0) {
+   sva += L2_SIZE;
+   continue;
+   }
+   pa = l2e & ~ATTR_MASK;
+   m = 

svn commit: r367281 - in head/sys: amd64/amd64 arm64/arm64 mips/mips powerpc/powerpc riscv/riscv

2020-11-02 Thread Alan Cox
Author: alc
Date: Mon Nov  2 19:20:06 2020
New Revision: 367281
URL: https://svnweb.freebsd.org/changeset/base/367281

Log:
  Tidy up the #includes.  Recent changes, such as the introduction of
  VM_ALLOC_WAITOK and vm_page_unwire_noq(), have eliminated the need for
  many of the #includes.
  
  Reviewed by:  kib, markj
  Differential Revision:https://reviews.freebsd.org/D27052

Modified:
  head/sys/amd64/amd64/uma_machdep.c
  head/sys/arm64/arm64/uma_machdep.c
  head/sys/mips/mips/uma_machdep.c
  head/sys/powerpc/powerpc/uma_machdep.c
  head/sys/riscv/riscv/uma_machdep.c

Modified: head/sys/amd64/amd64/uma_machdep.c
==
--- head/sys/amd64/amd64/uma_machdep.c  Mon Nov  2 18:48:06 2020
(r367280)
+++ head/sys/amd64/amd64/uma_machdep.c  Mon Nov  2 19:20:06 2020
(r367281)
@@ -30,15 +30,10 @@
 __FBSDID("$FreeBSD$");
 
 #include 
-#include 
 #include 
-#include 
-#include 
-#include 
 #include 
 #include 
 #include 
-#include 
 #include 
 #include 
 #include 

Modified: head/sys/arm64/arm64/uma_machdep.c
==
--- head/sys/arm64/arm64/uma_machdep.c  Mon Nov  2 18:48:06 2020
(r367280)
+++ head/sys/arm64/arm64/uma_machdep.c  Mon Nov  2 19:20:06 2020
(r367281)
@@ -28,11 +28,7 @@
 __FBSDID("$FreeBSD$");
 
 #include 
-#include 
 #include 
-#include 
-#include 
-#include 
 #include 
 #include 
 #include 

Modified: head/sys/mips/mips/uma_machdep.c
==
--- head/sys/mips/mips/uma_machdep.cMon Nov  2 18:48:06 2020
(r367280)
+++ head/sys/mips/mips/uma_machdep.cMon Nov  2 19:20:06 2020
(r367281)
@@ -30,11 +30,7 @@
 __FBSDID("$FreeBSD$");
 
 #include 
-#include 
 #include 
-#include 
-#include 
-#include 
 #include 
 #include 
 #include 
@@ -42,7 +38,6 @@ __FBSDID("$FreeBSD$");
 #include 
 #include 
 #include 
-#include 
 
 void *
 uma_small_alloc(uma_zone_t zone, vm_size_t bytes, int domain, u_int8_t *flags,

Modified: head/sys/powerpc/powerpc/uma_machdep.c
==
--- head/sys/powerpc/powerpc/uma_machdep.c  Mon Nov  2 18:48:06 2020
(r367280)
+++ head/sys/powerpc/powerpc/uma_machdep.c  Mon Nov  2 19:20:06 2020
(r367281)
@@ -28,21 +28,14 @@
 __FBSDID("$FreeBSD$");
 
 #include 
-#include 
-#include 
-#include 
-#include 
 #include 
+#include 
 #include 
-#include 
 #include 
 #include 
+#include 
 #include 
-#include 
-#include 
-#include 
 #include 
-#include 
 #include 
 #include 
 #include 

Modified: head/sys/riscv/riscv/uma_machdep.c
==
--- head/sys/riscv/riscv/uma_machdep.c  Mon Nov  2 18:48:06 2020
(r367280)
+++ head/sys/riscv/riscv/uma_machdep.c  Mon Nov  2 19:20:06 2020
(r367281)
@@ -28,19 +28,13 @@
 __FBSDID("$FreeBSD$");
 
 #include 
-#include 
 #include 
-#include 
-#include 
-#include 
 #include 
 #include 
 #include 
-#include 
 #include 
 #include 
 #include 
-#include 
 
 void *
 uma_small_alloc(uma_zone_t zone, vm_size_t bytes, int domain, u_int8_t *flags,
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


Re: svn commit: r367280 - head/lib/libc/gen

2020-11-02 Thread Oliver Pinter
On Monday, November 2, 2020, Stefan Eßer  wrote:

> Author: se
> Date: Mon Nov  2 18:48:06 2020
> New Revision: 367280
> URL: https://svnweb.freebsd.org/changeset/base/367280
>
> Log:
>   Re-arrange some of the code to separate writable user tree variables from
>   R/O variables.
>
>   While here fix some nearby style. No functional change intended.
>
>   MFC after:1 month


Is there any phabricator reference for this / these commit(s) + reviewer
lists?


trim
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


svn commit: r367280 - head/lib/libc/gen

2020-11-02 Thread Stefan Eßer
Author: se
Date: Mon Nov  2 18:48:06 2020
New Revision: 367280
URL: https://svnweb.freebsd.org/changeset/base/367280

Log:
  Re-arrange some of the code to separate writable user tree variables from
  R/O variables.
  
  While here fix some nearby style. No functional change intended.
  
  MFC after:1 month

Modified:
  head/lib/libc/gen/sysctl.c

Modified: head/lib/libc/gen/sysctl.c
==
--- head/lib/libc/gen/sysctl.c  Mon Nov  2 18:45:43 2020(r367279)
+++ head/lib/libc/gen/sysctl.c  Mon Nov  2 18:48:06 2020(r367280)
@@ -53,26 +53,42 @@ sysctl(const int *name, u_int namelen, void *oldp, siz
int retval;
size_t orig_oldlen;
 
-   orig_oldlen = oldlenp ? *oldlenp : 0;
+   orig_oldlen = oldlenp != NULL ? *oldlenp : 0;
retval = __sysctl(name, namelen, oldp, oldlenp, newp, newlen);
/*
-* All valid names under CTL_USER have a dummy entry in the sysctl
-* tree (to support name lookups and enumerations) with an
-* empty/zero value, and the true value is supplied by this routine.
-* For all such names, __sysctl() is used solely to validate the
-* name.
+* Valid names under CTL_USER except USER_LOCALBASE have a dummy entry
+* in the sysctl tree (to support name lookups and enumerations) with
+* an empty/zero value, and the true value is supplied by this routine.
+* For all such names, __sysctl() is used solely to validate the name.
 *
-* Return here unless there was a successful lookup for a CTL_USER
-* name.
+* Return here unless there was a successful lookup for a CTL_USER name.
 */
-   if (retval || name[0] != CTL_USER)
+   if (retval != 0 || name[0] != CTL_USER)
return (retval);
 
if (namelen != 2) {
errno = EINVAL;
return (-1);
}
-   if (newp != NULL && name[1] != USER_LOCALBASE) {
+
+   /* Variables under CLT_USER that may be overridden by kernel values */
+   switch (name[1]) {
+   case USER_LOCALBASE:
+   if (oldlenp == NULL || *oldlenp != 1)
+   return (0);
+   if (oldp != NULL) {
+   if (orig_oldlen < sizeof(_PATH_LOCALBASE)) {
+   errno = ENOMEM;
+   return (-1);
+   }
+   memmove(oldp, _PATH_LOCALBASE, sizeof(_PATH_LOCALBASE));
+   }
+   *oldlenp = sizeof(_PATH_LOCALBASE);
+   return (0);
+   }
+
+   /* Variables under CLT_USER whose values are immutably defined below */
+   if (newp != NULL) {
errno = EPERM;
return (-1);
}
@@ -87,26 +103,9 @@ sysctl(const int *name, u_int namelen, void *oldp, siz
if (oldp != NULL)
memmove(oldp, _PATH_STDPATH, sizeof(_PATH_STDPATH));
return (0);
-   case USER_LOCALBASE:
-   if (oldlenp != NULL) {
-   if (oldp == NULL) {
-   if (*oldlenp == 1)
-   *oldlenp = sizeof(_PATH_LOCALBASE);
-   } else {
-   if (*oldlenp != 1)
-   return (retval);
-   if (orig_oldlen < sizeof(_PATH_LOCALBASE)) {
-   errno = ENOMEM;
-   return (-1);
-   }
-   *oldlenp = sizeof(_PATH_LOCALBASE);
-   memmove(oldp, _PATH_LOCALBASE, 
sizeof(_PATH_LOCALBASE));
-   }
-   }
-   return (0);
}
 
-   if (oldp && *oldlenp < sizeof(int)) {
+   if (oldp != NULL && *oldlenp < sizeof(int)) {
errno = ENOMEM;
return (-1);
}
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


svn commit: r367279 - head/sys/compat/linux

2020-11-02 Thread Conrad Meyer
Author: cem
Date: Mon Nov  2 18:45:43 2020
New Revision: 367279
URL: https://svnweb.freebsd.org/changeset/base/367279

Log:
  linux(4): Quiesce unrecognized ioctl warning for F2FS query
  
  On Linux, sqlite probes for underlying F2FS filesystems that support
  certain kinds of atomic update with this ioctl.  The expected result on
  non-F2FS filesystem (i.e., all FreeBSD filesystems) is any error value.
  
  Minimally implement the ioctl and avoid the warning message.
  
  (This shows up in Linux Chrome, which embeds sqlite.)
  
  Reviewed by:  emaste, trasz
  Differential Revision:https://reviews.freebsd.org/D27050

Modified:
  head/sys/compat/linux/linux_ioctl.c
  head/sys/compat/linux/linux_ioctl.h

Modified: head/sys/compat/linux/linux_ioctl.c
==
--- head/sys/compat/linux/linux_ioctl.c Mon Nov  2 18:45:15 2020
(r367278)
+++ head/sys/compat/linux/linux_ioctl.c Mon Nov  2 18:45:43 2020
(r367279)
@@ -3631,6 +3631,7 @@ linux_ioctl_fallback(struct thread *td, struct linux_i
 
switch (args->cmd & 0x) {
case LINUX_BTRFS_IOC_CLONE:
+   case LINUX_F2FS_IOC_GET_FEATURES:
case LINUX_FS_IOC_FIEMAP:
return (ENOTSUP);
 

Modified: head/sys/compat/linux/linux_ioctl.h
==
--- head/sys/compat/linux/linux_ioctl.h Mon Nov  2 18:45:15 2020
(r367278)
+++ head/sys/compat/linux/linux_ioctl.h Mon Nov  2 18:45:43 2020
(r367279)
@@ -707,6 +707,11 @@
 #define LINUX_IOCTL_VIDEO2_MIN LINUX_VIDIOC_QUERYCAP
 #define LINUX_IOCTL_VIDEO2_MAX LINUX_VIDIOC_UNSUBSCRIBE_EVENT
 
+#defineLINUX_F2FS_IOC_GET_FEATURES 0xf50c /* 0x8004f50c */
+
+#defineLINUX_IOCTL_F2FS_MIN0xf500
+#defineLINUX_IOCTL_F2FS_MAXLINUX_F2FS_IOC_GET_FEATURES
+
 /*
  * Our libusb(8) calls emulated within linux(4).
  */
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


svn commit: r367278 - head/sys/compat/linux

2020-11-02 Thread Conrad Meyer
Author: cem
Date: Mon Nov  2 18:45:15 2020
New Revision: 367278
URL: https://svnweb.freebsd.org/changeset/base/367278

Log:
  linux(4): Deduplicate ioctl range construction with a helper macro
  
  No functional change.
  
  Reviewed by:  emaste, trasz
  Differential Revision:https://reviews.freebsd.org/D27049

Modified:
  head/sys/compat/linux/linux_ioctl.c
  head/sys/compat/linux/linux_ioctl.h

Modified: head/sys/compat/linux/linux_ioctl.c
==
--- head/sys/compat/linux/linux_ioctl.c Mon Nov  2 18:23:50 2020
(r367277)
+++ head/sys/compat/linux/linux_ioctl.c Mon Nov  2 18:45:15 2020
(r367278)
@@ -102,70 +102,35 @@ __FBSDID("$FreeBSD$");
 
 CTASSERT(LINUX_IFNAMSIZ == IFNAMSIZ);
 
-static linux_ioctl_function_t linux_ioctl_cdrom;
-static linux_ioctl_function_t linux_ioctl_vfat;
-static linux_ioctl_function_t linux_ioctl_console;
-static linux_ioctl_function_t linux_ioctl_hdio;
-static linux_ioctl_function_t linux_ioctl_disk;
-static linux_ioctl_function_t linux_ioctl_socket;
-static linux_ioctl_function_t linux_ioctl_sound;
-static linux_ioctl_function_t linux_ioctl_termio;
-static linux_ioctl_function_t linux_ioctl_private;
-static linux_ioctl_function_t linux_ioctl_drm;
-static linux_ioctl_function_t linux_ioctl_sg;
-static linux_ioctl_function_t linux_ioctl_v4l;
-static linux_ioctl_function_t linux_ioctl_v4l2;
-static linux_ioctl_function_t linux_ioctl_special;
-static linux_ioctl_function_t linux_ioctl_fbsd_usb;
-static linux_ioctl_function_t linux_ioctl_evdev;
-static linux_ioctl_function_t linux_ioctl_kcov;
+#defineDEFINE_LINUX_IOCTL_SET(shortname, SHORTNAME)\
+static linux_ioctl_function_t linux_ioctl_ ## shortname;   \
+static struct linux_ioctl_handler shortname ## _handler = {\
+   .func = linux_ioctl_ ## shortname,  \
+   .low = LINUX_IOCTL_ ## SHORTNAME ## _MIN,   \
+   .high = LINUX_IOCTL_ ## SHORTNAME ## _MAX,  \
+}; \
+DATA_SET(linux_ioctl_handler_set, shortname ## _handler)
 
-static struct linux_ioctl_handler cdrom_handler =
-{ linux_ioctl_cdrom, LINUX_IOCTL_CDROM_MIN, LINUX_IOCTL_CDROM_MAX };
-static struct linux_ioctl_handler vfat_handler =
-{ linux_ioctl_vfat, LINUX_IOCTL_VFAT_MIN, LINUX_IOCTL_VFAT_MAX };
-static struct linux_ioctl_handler console_handler =
-{ linux_ioctl_console, LINUX_IOCTL_CONSOLE_MIN, LINUX_IOCTL_CONSOLE_MAX };
-static struct linux_ioctl_handler hdio_handler =
-{ linux_ioctl_hdio, LINUX_IOCTL_HDIO_MIN, LINUX_IOCTL_HDIO_MAX };
-static struct linux_ioctl_handler disk_handler =
-{ linux_ioctl_disk, LINUX_IOCTL_DISK_MIN, LINUX_IOCTL_DISK_MAX };
-static struct linux_ioctl_handler socket_handler =
-{ linux_ioctl_socket, LINUX_IOCTL_SOCKET_MIN, LINUX_IOCTL_SOCKET_MAX };
-static struct linux_ioctl_handler sound_handler =
-{ linux_ioctl_sound, LINUX_IOCTL_SOUND_MIN, LINUX_IOCTL_SOUND_MAX };
-static struct linux_ioctl_handler private_handler =
-{ linux_ioctl_private, LINUX_IOCTL_PRIVATE_MIN, LINUX_IOCTL_PRIVATE_MAX };
-static struct linux_ioctl_handler drm_handler =
-{ linux_ioctl_drm, LINUX_IOCTL_DRM_MIN, LINUX_IOCTL_DRM_MAX };
-static struct linux_ioctl_handler sg_handler =
-{ linux_ioctl_sg, LINUX_IOCTL_SG_MIN, LINUX_IOCTL_SG_MAX };
-static struct linux_ioctl_handler video_handler =
-{ linux_ioctl_v4l, LINUX_IOCTL_VIDEO_MIN, LINUX_IOCTL_VIDEO_MAX };
-static struct linux_ioctl_handler video2_handler =
-{ linux_ioctl_v4l2, LINUX_IOCTL_VIDEO2_MIN, LINUX_IOCTL_VIDEO2_MAX };
-static struct linux_ioctl_handler fbsd_usb =
-{ linux_ioctl_fbsd_usb, FBSD_LUSB_MIN, FBSD_LUSB_MAX };
-static struct linux_ioctl_handler evdev_handler =
-{ linux_ioctl_evdev, LINUX_IOCTL_EVDEV_MIN, LINUX_IOCTL_EVDEV_MAX };
-static struct linux_ioctl_handler kcov_handler =
-{ linux_ioctl_kcov, LINUX_KCOV_MIN, LINUX_KCOV_MAX };
+DEFINE_LINUX_IOCTL_SET(cdrom, CDROM);
+DEFINE_LINUX_IOCTL_SET(vfat, VFAT);
+DEFINE_LINUX_IOCTL_SET(console, CONSOLE);
+DEFINE_LINUX_IOCTL_SET(hdio, HDIO);
+DEFINE_LINUX_IOCTL_SET(disk, DISK);
+DEFINE_LINUX_IOCTL_SET(socket, SOCKET);
+DEFINE_LINUX_IOCTL_SET(sound, SOUND);
+DEFINE_LINUX_IOCTL_SET(termio, TERMIO);
+DEFINE_LINUX_IOCTL_SET(private, PRIVATE);
+DEFINE_LINUX_IOCTL_SET(drm, DRM);
+DEFINE_LINUX_IOCTL_SET(sg, SG);
+DEFINE_LINUX_IOCTL_SET(v4l, VIDEO);
+DEFINE_LINUX_IOCTL_SET(v4l2, VIDEO2);
+DEFINE_LINUX_IOCTL_SET(fbsd_usb, FBSD_LUSB);
+DEFINE_LINUX_IOCTL_SET(evdev, EVDEV);
+DEFINE_LINUX_IOCTL_SET(kcov, KCOV);
 
-DATA_SET(linux_ioctl_handler_set, cdrom_handler);
-DATA_SET(linux_ioctl_handler_set, vfat_handler);
-DATA_SET(linux_ioctl_handler_set, console_handler);
-DATA_SET(linux_ioctl_handler_set, hdio_handler);
-DATA_SET(linux_ioctl_handler_set, disk_handler);
-DATA_SET(linux_ioctl_handler_set, socket_handler);
-DATA_SET(linux_ioctl_handler_set, sound_handler);
-DATA_SET(linux_ioctl_handler_set, private_handler);

svn commit: r367277 - head

2020-11-02 Thread Emmanuel Vadot
Author: manu
Date: Mon Nov  2 18:23:50 2020
New Revision: 367277
URL: https://svnweb.freebsd.org/changeset/base/367277

Log:
  pkgbase: Add incremental packages
  
  This adds a new target update-packages which will create the new packages
  compared to the last run.
  
  This is how to use it:
  At this point we cut a release
  $ make buildworld ...
  $ make buildkernel
  $ make packages
  
  There is now a PKG_VERSION directory with latest link pointing to it
  Distribute the packages to server
  
  $ something something that update the source tree
  $ make buildworld ...
  $ make buildkernel
  $ make update-packages
  You know have a PKG_VERSION directory in the REPODIR and latest link pointing 
to it.
  In PKG_VERSION dir only the packages which differs from the latest run are
  named PKG_VERSION, otherwise the old packages are there.
  
  The process is :
  Build the new packages in the PKG_VERSION directory
  Compare the internal data with the PKG_VERSION_FROM version. The comparison 
is done
  by checking the internal hash of the packages.
  By default PKG_VERSION_FROM is set to what the latest link points to.
  If the old and new version matches, we rm the new package and cp the old one.
  
  Differential Revision:https://reviews.freebsd.org/D25984

Modified:
  head/Makefile
  head/Makefile.inc1

Modified: head/Makefile
==
--- head/Makefile   Mon Nov  2 17:39:59 2020(r367276)
+++ head/Makefile   Mon Nov  2 18:23:50 2020(r367277)
@@ -162,8 +162,8 @@ TGTS=   all all-man buildenv buildenvvars buildkernel bu
xdev-links native-xtools native-xtools-install stageworld stagekernel \
stage-packages stage-packages-kernel stage-packages-world \
create-packages-world create-packages-kernel create-packages \
-   packages installconfig real-packages sign-packages package-pkg \
-   print-dir test-system-compiler test-system-linker
+   update-packages packages installconfig real-packages 
real-update-packages \
+   sign-packages package-pkg print-dir test-system-compiler 
test-system-linker
 
 # These targets require a TARGET and TARGET_ARCH be defined.
 XTGTS= native-xtools native-xtools-install xdev xdev-build xdev-install \

Modified: head/Makefile.inc1
==
--- head/Makefile.inc1  Mon Nov  2 17:39:59 2020(r367276)
+++ head/Makefile.inc1  Mon Nov  2 18:23:50 2020(r367277)
@@ -568,12 +568,12 @@ EXTRA_REVISION= _${_BRANCH:C/-PRERELEASE/.p/}
 .elif ${_BRANCH:M*-p*}
 EXTRA_REVISION=_${_BRANCH:C/.*-p([0-9]+$)/\1/}
 .endif
-PKG_VERSION=   ${_REVISION}${EXTRA_REVISION}
+PKG_VERSION:=  ${_REVISION}${EXTRA_REVISION}
 .endif
 .endif # !defined(PKG_VERSION)
 
 .if !defined(PKG_TIMESTAMP)
-TIMEEPOCHNOW=  %s
+TIMEEPOCHNOW=  %s
 SOURCE_DATE_EPOCH= ${TIMEEPOCHNOW:gmtime}
 .else
 SOURCE_DATE_EPOCH= ${PKG_TIMESTAMP}
@@ -1855,9 +1855,22 @@ _pkgbootstrap: .PHONY
@env ASSUME_ALWAYS_YES=YES pkg bootstrap
 .endif
 
+.if make(create-world-packages-jobs) || make(create-kernel-packages*) || 
make(real-update-packages)
+PKG_ABI!=${PKG_CMD} -o ABI_FILE=${WSTAGEDIR}/usr/bin/uname config ABI
+.endif
+
+.if !defined(PKG_VERSION_FROM)
+.if defined(PKG_ABI)
+PKG_VERSION_FROM!=/usr/bin/readlink ${REPODIR}/${PKG_ABI}/latest
+.endif
+.endif
+
 packages: .PHONY
${_+_}${MAKE} -C ${.CURDIR} PKG_VERSION=${PKG_VERSION} real-packages
 
+update-packages: .PHONY
+   ${_+_}${MAKE} -C ${.CURDIR} PKG_VERSION=${PKG_VERSION} 
real-update-packages
+
 package-pkg: .PHONY
rm -rf /tmp/ports.${TARGET} || :
env ${WMAKEENV:Q} SRCDIR=${.CURDIR} PORTSDIR=${PORTSDIR} 
REVISION=${_REVISION} \
@@ -1867,6 +1880,28 @@ package-pkg: .PHONY
 
 real-packages: stage-packages create-packages sign-packages .PHONY
 
+real-update-packages: stage-packages .PHONY
+   ${_+_}${MAKE} -C ${.CURDIR} PKG_VERSION=${PKG_VERSION} create-packages
+   @echo "==> Checking for new packages (comparing ${PKG_VERSION} to 
${PKG_VERSION_FROM})"
+   @for pkg in 
${REPODIR}/${PKG_ABI}/${PKG_VERSION_FROM}/${PKG_NAME_PREFIX}-*; do \
+ pkgname=$$(pkg query -F $${pkg} '%n' | sed 
's/${PKG_NAME_PREFIX}-\(.*\)/\1/') ; \
+ 
newpkgname=${PKG_NAME_PREFIX}-$${pkgname}-${PKG_VERSION}.${PKG_FORMAT} ; \
+ oldsum=$$(pkg query -F $${pkg} '%X') ; \
+ if [ ! -f ${REPODIR}/${PKG_ABI}/${PKG_VERSION}/$${newpkgname} ]; then 
\
+   continue; \
+ fi ; \
+ newsum=$$(pkg query -F 
${REPODIR}/${PKG_ABI}/${PKG_VERSION}/$${newpkgname} '%X') ; \
+ if [ "$${oldsum}" == "$${newsum}" ]; then \
+  echo "==> Keeping old 
${PKG_NAME_PREFIX}-$${pkgname}-${PKG_VERSION_FROM}.${PKG_FORMAT}" ; \
+  rm ${REPODIR}/${PKG_ABI}/${PKG_VERSION}/$${newpkgname} ; \
+  cp $${pkg} ${REPODIR}/${PKG_ABI}/${PKG_VERSION} ; \
+ else \
+  

svn commit: r367275 - head/sys/kern

2020-11-02 Thread Mateusz Guzik
Author: mjg
Date: Mon Nov  2 17:39:15 2020
New Revision: 367275
URL: https://svnweb.freebsd.org/changeset/base/367275

Log:
  malloc: prefix zones with malloc-
  
  Reviewed by:  markj
  Differential Revision:https://reviews.freebsd.org/D27038

Modified:
  head/sys/kern/kern_malloc.c

Modified: head/sys/kern/kern_malloc.c
==
--- head/sys/kern/kern_malloc.c Mon Nov  2 17:38:08 2020(r367274)
+++ head/sys/kern/kern_malloc.c Mon Nov  2 17:39:15 2020(r367275)
@@ -158,19 +158,19 @@ struct {
const char *kz_name;
uma_zone_t kz_zone[MALLOC_DEBUG_MAXZONES];
 } kmemzones[] = {
-   {16, "16", },
-   {32, "32", },
-   {64, "64", },
-   {128, "128", },
-   {256, "256", },
-   {512, "512", },
-   {1024, "1024", },
-   {2048, "2048", },
-   {4096, "4096", },
-   {8192, "8192", },
-   {16384, "16384", },
-   {32768, "32768", },
-   {65536, "65536", },
+   {16, "malloc-16", },
+   {32, "malloc-32", },
+   {64, "malloc-64", },
+   {128, "malloc-128", },
+   {256, "malloc-256", },
+   {512, "malloc-512", },
+   {1024, "malloc-1024", },
+   {2048, "malloc-2048", },
+   {4096, "malloc-4096", },
+   {8192, "malloc-8192", },
+   {16384, "malloc-16384", },
+   {32768, "malloc-32768", },
+   {65536, "malloc-65536", },
{0, NULL},
 };
 
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


svn commit: r367276 - head/sys/contrib/openzfs/module/zstd

2020-11-02 Thread Mateusz Guzik
Author: mjg
Date: Mon Nov  2 17:39:59 2020
New Revision: 367276
URL: https://svnweb.freebsd.org/changeset/base/367276

Log:
  zfs: zstd: short-circuit cleaning buffers if none exist
  
  This avoids a barrage of locking every minute.

Modified:
  head/sys/contrib/openzfs/module/zstd/zfs_zstd.c

Modified: head/sys/contrib/openzfs/module/zstd/zfs_zstd.c
==
--- head/sys/contrib/openzfs/module/zstd/zfs_zstd.c Mon Nov  2 17:39:15 
2020(r367275)
+++ head/sys/contrib/openzfs/module/zstd/zfs_zstd.c Mon Nov  2 17:39:59 
2020(r367276)
@@ -700,6 +700,13 @@ zstd_mempool_deinit(void)
 void
 zfs_zstd_cache_reap_now(void)
 {
+
+   /*
+* Short-circuit if there are no buffers to begin with.
+*/
+   if (ZSTDSTAT(zstd_stat_buffers) == 0)
+   return;
+
/*
 * calling alloc with zero size seeks
 * and releases old unused objects
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


svn commit: r367274 - in head: lib/libmemstat share/man/man9 sys/kern usr.bin/vmstat

2020-11-02 Thread Mateusz Guzik
Author: mjg
Date: Mon Nov  2 17:38:08 2020
New Revision: 367274
URL: https://svnweb.freebsd.org/changeset/base/367274

Log:
  malloc: export kernel zones instead of relying on them being power-of-2
  
  Reviewed by:  markj (previous version)
  Differential Revision:https://reviews.freebsd.org/D27026

Modified:
  head/lib/libmemstat/memstat.h
  head/lib/libmemstat/memstat_malloc.c
  head/share/man/man9/malloc.9
  head/sys/kern/kern_malloc.c
  head/usr.bin/vmstat/vmstat.c

Modified: head/lib/libmemstat/memstat.h
==
--- head/lib/libmemstat/memstat.h   Mon Nov  2 15:01:37 2020
(r367273)
+++ head/lib/libmemstat/memstat.h   Mon Nov  2 17:38:08 2020
(r367274)
@@ -118,6 +118,13 @@ intmemstat_kvm_malloc(struct memory_type_list 
*list, 
 intmemstat_kvm_uma(struct memory_type_list *list, void *kvm_handle);
 
 /*
+ * General malloc routines.
+ */
+size_t memstat_malloc_zone_get_count(void);
+size_t memstat_malloc_zone_get_size(size_t n);
+intmemstat_malloc_zone_used(const struct memory_type *mtp, size_t n);
+
+/*
  * Accessor methods for struct memory_type.
  */
 const char *memstat_get_name(const struct memory_type *mtp);

Modified: head/lib/libmemstat/memstat_malloc.c
==
--- head/lib/libmemstat/memstat_malloc.cMon Nov  2 15:01:37 2020
(r367273)
+++ head/lib/libmemstat/memstat_malloc.cMon Nov  2 17:38:08 2020
(r367274)
@@ -44,10 +44,22 @@
 #include "memstat.h"
 #include "memstat_internal.h"
 
+static int memstat_malloc_zone_count;
+static int memstat_malloc_zone_sizes[32];
+
+static int memstat_malloc_zone_init(void);
+static int memstat_malloc_zone_init_kvm(kvm_t *kvm);
+
 static struct nlist namelist[] = {
 #defineX_KMEMSTATISTICS0
{ .n_name = "_kmemstatistics" },
-#defineX_MP_MAXCPUS1
+#defineX_KMEMZONES 1
+   { .n_name = "_kmemzones" },
+#defineX_NUMZONES  2
+   { .n_name = "_numzones" },
+#defineX_VM_MALLOC_ZONE_COUNT  3
+   { .n_name = "_vm_malloc_zone_count" },
+#defineX_MP_MAXCPUS4
{ .n_name = "_mp_maxcpus" },
{ .n_name = "" },
 };
@@ -111,6 +123,11 @@ retry:
return (-1);
}
 
+   if (memstat_malloc_zone_init() == -1) {
+   list->mtl_error = MEMSTAT_ERROR_VERSION;
+   return (-1);
+   }
+
size = sizeof(*mthp) + count * (sizeof(*mthp) + sizeof(*mtsp) *
maxcpus);
 
@@ -333,6 +350,12 @@ memstat_kvm_malloc(struct memory_type_list *list, void
return (-1);
}
 
+   ret = memstat_malloc_zone_init_kvm(kvm);
+   if (ret != 0) {
+   list->mtl_error = ret;
+   return (-1);
+   }
+
mp_ncpus = kvm_getncpus(kvm);
 
for (typep = kmemstatistics; typep != NULL; typep = type.ks_next) {
@@ -413,6 +436,112 @@ memstat_kvm_malloc(struct memory_type_list *list, void
mtp->mt_bytes = mtp->mt_memalloced - mtp->mt_memfreed;
mtp->mt_count = mtp->mt_numallocs - mtp->mt_numfrees;
}
+
+   return (0);
+}
+
+static int
+memstat_malloc_zone_init(void)
+{
+   size_t size;
+
+   size = sizeof(memstat_malloc_zone_count);
+   if (sysctlbyname("vm.malloc.zone_count", _malloc_zone_count,
+   , NULL, 0) < 0) {
+   return (-1);
+   }
+
+   if (memstat_malloc_zone_count > (int)nitems(memstat_malloc_zone_sizes)) 
{
+   return (-1);
+   }
+
+   size = sizeof(memstat_malloc_zone_sizes);
+   if (sysctlbyname("vm.malloc.zone_sizes", _malloc_zone_sizes,
+   , NULL, 0) < 0) {
+   return (-1);
+   }
+
+   return (0);
+}
+
+/*
+ * Copied from kern_malloc.c
+ *
+ * kz_zone is an array sized at compilation time, the size is exported in
+ * "numzones". Below we need to iterate kz_size.
+ */
+struct memstat_kmemzone {
+   int kz_size;
+   const char *kz_name;
+   void *kz_zone[1];
+};
+
+static int
+memstat_malloc_zone_init_kvm(kvm_t *kvm)
+{
+   struct memstat_kmemzone *kmemzones, *kz;
+   int numzones, objsize, allocsize, ret;
+   int i;
+
+   ret = kread_symbol(kvm, X_VM_MALLOC_ZONE_COUNT,
+   _malloc_zone_count, sizeof(memstat_malloc_zone_count), 0);
+   if (ret != 0) {
+   return (ret);
+   }
+
+   ret = kread_symbol(kvm, X_NUMZONES, , sizeof(numzones), 0);
+   if (ret != 0) {
+   return (ret);
+   }
+
+   objsize = __offsetof(struct memstat_kmemzone, kz_zone) +
+   sizeof(void *) * numzones;
+
+   allocsize = objsize * memstat_malloc_zone_count;
+   kmemzones = malloc(allocsize);
+   if (kmemzones == NULL) {
+   return (MEMSTAT_ERROR_NOMEMORY);
+   }
+   ret = kread_symbol(kvm, 

svn commit: r367273 - head/lib/libnetmap

2020-11-02 Thread Adrian Chadd
Author: adrian
Date: Mon Nov  2 15:01:37 2020
New Revision: 367273
URL: https://svnweb.freebsd.org/changeset/base/367273

Log:
  [libnetmap] Fix 32 bit compilation under gcc-6.4
  
  Use uintptr_t to cast a uint64_t to a pointer type.
  Yeah, it isn't technically correct for platforms with pointers
  > 64 bits, but it's fine here.
  
  This fixes 32 bit compat library builds on amd64 and also
  mips32 builds.
  
  Reviewed by:  imp
  Differential Revision:https://reviews.freebsd.org/D26790

Modified:
  head/lib/libnetmap/nmreq.c

Modified: head/lib/libnetmap/nmreq.c
==
--- head/lib/libnetmap/nmreq.c  Mon Nov  2 14:30:55 2020(r367272)
+++ head/lib/libnetmap/nmreq.c  Mon Nov  2 15:01:37 2020(r367273)
@@ -603,10 +603,9 @@ nmreq_options_decode(const char *opt, struct nmreq_opt
 struct nmreq_option *
 nmreq_find_option(struct nmreq_header *h, uint32_t t)
 {
-   struct nmreq_option *o;
+   struct nmreq_option *o = NULL;
 
-   for (o = (struct nmreq_option *)h->nr_options; o != NULL;
-   o = (struct nmreq_option *)o->nro_next) {
+   nmreq_foreach_option(h, o) {
if (o->nro_reqtype == t)
break;
}
@@ -633,8 +632,14 @@ nmreq_free_options(struct nmreq_header *h)
 {
struct nmreq_option *o, *next;
 
-   for (o = (struct nmreq_option *)h->nr_options; o != NULL; o = next) {
-   next = (struct nmreq_option *)o->nro_next;
+   /*
+* Note: can't use nmreq_foreach_option() here; it frees the
+* list as it's walking and nmreq_foreach_option() isn't
+* modification-safe.
+*/
+   for (o = (struct nmreq_option *)(uintptr_t)h->nr_options; o != NULL;
+   o = next) {
+   next = (struct nmreq_option *)(uintptr_t)o->nro_next;
free(o);
}
 }
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


svn commit: r367269 - head/sys/ofed/drivers/infiniband/core

2020-11-02 Thread Hans Petter Selasky
Author: hselasky
Date: Mon Nov  2 10:44:29 2020
New Revision: 367269
URL: https://svnweb.freebsd.org/changeset/base/367269

Log:
  Fix for referencing file via its vnode in ibore.
  
  Use the native vnode lookup functions, instead of going via the LinuxKPI,
  because the file referenced is typically created outside the LinuxKPI, and
  the LinuxKPI's fdget() can only resolve file descriptor numbers which
  were created by itself.
  
  The vnode pointer is used as an identifier to identify XRCD handles which
  are sharing resources.
  
  This patch fixes the so-called XRCD support in ibcore for FreeBSD.
  Refer to ibv_open_xrcd(3) for more information how the file descriptor
  argument is used.
  
  Reviewed by:  kib@
  MFC after:1 week
  Sponsored by: Mellanox Technologies // NVIDIA Networking

Modified:
  head/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c

Modified: head/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c
==
--- head/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c   Mon Nov  2 
08:26:19 2020(r367268)
+++ head/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c   Mon Nov  2 
10:44:29 2020(r367269)
@@ -666,11 +666,11 @@ err_put:
 struct xrcd_table_entry {
struct rb_node  node;
struct ib_xrcd *xrcd;
-   struct inode   *inode;
+   struct vnode   *vnode;
 };
 
 static int xrcd_table_insert(struct ib_uverbs_device *dev,
-   struct inode *inode,
+   struct vnode *vnode,
struct ib_xrcd *xrcd)
 {
struct xrcd_table_entry *entry, *scan;
@@ -682,15 +682,15 @@ static int xrcd_table_insert(struct ib_uverbs_device *
return -ENOMEM;
 
entry->xrcd  = xrcd;
-   entry->inode = inode;
+   entry->vnode = vnode;
 
while (*p) {
parent = *p;
scan = rb_entry(parent, struct xrcd_table_entry, node);
 
-   if (inode < scan->inode) {
+   if ((uintptr_t)vnode < (uintptr_t)scan->vnode) {
p = &(*p)->rb_left;
-   } else if (inode > scan->inode) {
+   } else if ((uintptr_t)vnode > (uintptr_t)scan->vnode) {
p = &(*p)->rb_right;
} else {
kfree(entry);
@@ -700,12 +700,12 @@ static int xrcd_table_insert(struct ib_uverbs_device *
 
rb_link_node(>node, parent, p);
rb_insert_color(>node, >xrcd_tree);
-   igrab(inode);
+   vrefact(vnode);
return 0;
 }
 
 static struct xrcd_table_entry *xrcd_table_search(struct ib_uverbs_device *dev,
- struct inode *inode)
+ struct vnode *vnode)
 {
struct xrcd_table_entry *entry;
struct rb_node *p = dev->xrcd_tree.rb_node;
@@ -713,9 +713,9 @@ static struct xrcd_table_entry *xrcd_table_search(stru
while (p) {
entry = rb_entry(p, struct xrcd_table_entry, node);
 
-   if (inode < entry->inode)
+   if ((uintptr_t)vnode < (uintptr_t)entry->vnode)
p = p->rb_left;
-   else if (inode > entry->inode)
+   else if ((uintptr_t)vnode > (uintptr_t)entry->vnode)
p = p->rb_right;
else
return entry;
@@ -724,11 +724,11 @@ static struct xrcd_table_entry *xrcd_table_search(stru
return NULL;
 }
 
-static struct ib_xrcd *find_xrcd(struct ib_uverbs_device *dev, struct inode 
*inode)
+static struct ib_xrcd *find_xrcd(struct ib_uverbs_device *dev, struct vnode 
*vnode)
 {
struct xrcd_table_entry *entry;
 
-   entry = xrcd_table_search(dev, inode);
+   entry = xrcd_table_search(dev, vnode);
if (!entry)
return NULL;
 
@@ -736,13 +736,13 @@ static struct ib_xrcd *find_xrcd(struct ib_uverbs_devi
 }
 
 static void xrcd_table_delete(struct ib_uverbs_device *dev,
- struct inode *inode)
+ struct vnode *vnode)
 {
struct xrcd_table_entry *entry;
 
-   entry = xrcd_table_search(dev, inode);
+   entry = xrcd_table_search(dev, vnode);
if (entry) {
-   iput(inode);
+   vrele(vnode);
rb_erase(>node, >xrcd_tree);
kfree(entry);
}
@@ -758,8 +758,7 @@ ssize_t ib_uverbs_open_xrcd(struct ib_uverbs_file *fil
struct ib_udata udata;
struct ib_uxrcd_object *obj;
struct ib_xrcd *xrcd = NULL;
-   struct fd   f = {NULL};
-   struct inode   *inode = NULL;
+   struct vnode   *vnode = NULL;
int ret = 0;
int new_xrcd = 0;
 
@@ -777,14 +776,11 @@ ssize_t 

svn commit: r367268 - head/sys/arm64/arm64

2020-11-02 Thread Michal Meloun
Author: mmel
Date: Mon Nov  2 08:26:19 2020
New Revision: 367268
URL: https://svnweb.freebsd.org/changeset/base/367268

Log:
  Improve loading of multipage aligned buffers.
  
  The multipage alignment requirements is incompatible with many aspects
  of actual busdma code. Multi-page alignment requests are incompatible
  with many aspects of current busdma code. Mainly with partially bounced
  buffer segments and per-page loop in bus_dmamap_load_buffer(). Because
  proper implementation would be a major restructuring of the code, add
  the fix only for already known uses and do KASSERT for all other cases.
  
  For this reason, bus_dmamap_load_buffer () should take the memory allocated
  by bus_dmam_alloc () as one segment bypassing per page segmentation. We can
  do this because it is guaranteed that the memory is physically continuous.
  
  Reviewed by:  bz
  Tested by:imp, mv, daniel.engberg.lists_pyret.net, kjopek_gmail.com
  Differential Revision: https://reviews.freebsd.org/D26735

Modified:
  head/sys/arm64/arm64/busdma_bounce.c

Modified: head/sys/arm64/arm64/busdma_bounce.c
==
--- head/sys/arm64/arm64/busdma_bounce.cMon Nov  2 06:16:11 2020
(r367267)
+++ head/sys/arm64/arm64/busdma_bounce.cMon Nov  2 08:26:19 2020
(r367268)
@@ -501,13 +501,6 @@ static int
 bounce_bus_dmamem_alloc(bus_dma_tag_t dmat, void** vaddr, int flags,
 bus_dmamap_t *mapp)
 {
-   /*
-* XXX ARM64TODO:
-* This bus_dma implementation requires IO-Coherent architecutre.
-* If IO-Coherency is not guaranteed, the BUS_DMA_COHERENT flag has
-* to be implented using non-cacheable memory.
-*/
-
vm_memattr_t attr;
int mflags;
 
@@ -830,7 +823,19 @@ bounce_bus_dmamap_load_phys(bus_dma_tag_t dmat, bus_dm
sgsize = MIN(buflen, dmat->common.maxsegsz);
if (map->pagesneeded != 0 &&
must_bounce(dmat, map, curaddr, sgsize)) {
-   sgsize = MIN(sgsize, PAGE_SIZE - (curaddr & PAGE_MASK));
+   /*
+* The attempt to split a physically continuous buffer
+* seems very controversial, it's unclear whether we
+* can do this in all cases. Also, memory for bounced
+* buffers is allocated as pages, so we cannot
+* guarantee multipage alignment.
+*/
+   KASSERT(dmat->common.alignment <= PAGE_SIZE,
+   ("bounced buffer cannot have alignment bigger "
+   "than PAGE_SIZE: %lu", dmat->common.alignment));
+   sgsize = PAGE_SIZE - (curaddr & PAGE_MASK);
+   sgsize = roundup2(sgsize, dmat->common.alignment);
+   sgsize = MIN(sgsize, dmat->common.maxsegsz);
curaddr = add_bounce_page(dmat, map, 0, curaddr,
sgsize);
} else if ((map->flags & DMAMAP_COHERENT) == 0) {
@@ -843,11 +848,11 @@ bounce_bus_dmamap_load_phys(bus_dma_tag_t dmat, bus_dm
sl++;
sl->vaddr = 0;
sl->paddr = curaddr;
-   sl->datacount = sgsize;
sl->pages = PHYS_TO_VM_PAGE(curaddr);
KASSERT(sl->pages != NULL,
("%s: page at PA:0x%08lx is not in "
"vm_page_array", __func__, curaddr));
+   sl->datacount = sgsize;
} else
sl->datacount += sgsize;
}
@@ -880,6 +885,11 @@ bounce_bus_dmamap_load_buffer(bus_dma_tag_t dmat, bus_
vm_offset_t kvaddr, vaddr, sl_vend;
int error;
 
+   KASSERT((map->flags & DMAMAP_FROM_DMAMEM) != 0 ||
+   dmat->common.alignment <= PAGE_SIZE,
+   ("loading user buffer with alignment bigger than PAGE_SIZE is not "
+   "supported"));
+
if (segs == NULL)
segs = dmat->segments;
 
@@ -895,6 +905,11 @@ bounce_bus_dmamap_load_buffer(bus_dma_tag_t dmat, bus_
}
}
 
+   /*
+* XXX Optimally we should parse input buffer for physically
+* continuous segments first and then pass these segment into
+* load loop.
+*/
sl = map->slist + map->sync_count - 1;
vaddr = (vm_offset_t)buf;
sl_pend = 0;
@@ -916,15 +931,25 @@ bounce_bus_dmamap_load_buffer(bus_dma_tag_t dmat, bus_
 * Compute the segment size, and adjust counts.
 */
max_sgsize = MIN(buflen, dmat->common.maxsegsz);
-   sgsize = PAGE_SIZE - (curaddr & PAGE_MASK);
+   if ((map->flags &