svn commit: r351629 - head/sys/net

2019-08-30 Thread Matt Joras 
Author: mjoras
Date: Fri Aug 30 20:19:43 2019
New Revision: 351629
URL: https://svnweb.freebsd.org/changeset/base/351629

Log:
  Wrap a vlan's parent's if_output in a separate function.
  
  When a vlan interface is created, its if_output is set directly to the
  parent interface's if_output. This is fine in the normal case but has an
  unfortunate consequence if you end up with a certain combination of vlan
  and lagg interfaces.
  
  Consider you have a lagg interface with a single laggport member. When
  an interface is added to a lagg its if_output is set to
  lagg_port_output, which blackholes traffic from the normal networking
  stack but not certain frames from BPF (pseudo_AF_HDRCMPLT). If you now
  create a vlan with the laggport member (not the lagg interface) as its
  parent, its if_output is set to lagg_port_output as well. While this is
  confusing conceptually and likely represents a misconfigured system, it
  is not itself a problem. The problem arises when you then remove the
  lagg interface. Doing this resets the if_output of the laggport member
  back to its original state, but the vlan's if_output is left pointing to
  lagg_port_output. This gives rise to the possibility that the system
  will panic when e.g. bpf is used to send any frames on the vlan
  interface.
  
  Fix this by creating a new function, vlan_output, which simply wraps the
  parent's current if_output. That way when the parent's if_output is
  restored there is no stale usage of lagg_port_output.
  
  Reviewed by:  rstone
  Differential Revision:D21209

Modified:
  head/sys/net/if_vlan.c

Modified: head/sys/net/if_vlan.c
==
--- head/sys/net/if_vlan.c  Fri Aug 30 19:35:44 2019(r351628)
+++ head/sys/net/if_vlan.c  Fri Aug 30 20:19:43 2019(r351629)
@@ -294,6 +294,8 @@ static  int vlan_setflag(struct ifnet *ifp, int flag, i
 static int vlan_setflags(struct ifnet *ifp, int status);
 static int vlan_setmulti(struct ifnet *ifp);
 static int vlan_transmit(struct ifnet *ifp, struct mbuf *m);
+static int vlan_output(struct ifnet *ifp, struct mbuf *m,
+const struct sockaddr *dst, struct route *ro);
 static void vlan_unconfig(struct ifnet *ifp);
 static void vlan_unconfig_locked(struct ifnet *ifp, int departing);
 static int vlan_config(struct ifvlan *ifv, struct ifnet *p, uint16_t tag);
@@ -1209,6 +1211,27 @@ vlan_transmit(struct ifnet *ifp, struct mbuf *m)
return (error);
 }
 
+static int
+vlan_output(struct ifnet *ifp, struct mbuf *m, const struct sockaddr *dst,
+struct route *ro)
+{
+   struct epoch_tracker et;
+   struct ifvlan *ifv;
+   struct ifnet *p;
+
+   NET_EPOCH_ENTER(et);
+   ifv = ifp->if_softc;
+   if (TRUNK(ifv) == NULL) {
+   NET_EPOCH_EXIT(et);
+   m_freem(m);
+   return (ENETDOWN);
+   }
+   p = PARENT(ifv);
+   NET_EPOCH_EXIT(et);
+   return p->if_output(ifp, m, dst, ro);
+}
+
+
 /*
  * The ifp->if_qflush entry point for vlan(4) is a no-op.
  */
@@ -1424,12 +1447,17 @@ vlan_config(struct ifvlan *ifv, struct ifnet *p, uint1
 */
ifp->if_mtu = p->if_mtu - ifv->ifv_mtufudge;
ifp->if_baudrate = p->if_baudrate;
-   ifp->if_output = p->if_output;
ifp->if_input = p->if_input;
ifp->if_resolvemulti = p->if_resolvemulti;
ifp->if_addrlen = p->if_addrlen;
ifp->if_broadcastaddr = p->if_broadcastaddr;
ifp->if_pcp = ifv->ifv_pcp;
+
+   /*
+* We wrap the parent's if_output using vlan_output to ensure that it
+* can't become stale.
+*/
+   ifp->if_output = vlan_output;
 
/*
 * Copy only a selected subset of flags from the parent.
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r334609 - head/usr.sbin/pmc

2018-06-04 Thread Matt Joras 
On Mon, Jun 4, 2018 at 8:18 AM, Rodney W. Grimes
 wrote:
> [ Charset UTF-8 unsupported, converting... ]
>> Author: mmacy
>> Date: Mon Jun  4 06:30:35 2018
>> New Revision: 334609
>> URL: https://svnweb.freebsd.org/changeset/base/334609
>>
>> Log:
>>   pmc filter: avoid spurious gcc uninitialized warning
>
> Is it truely uninitialzized, or are you silencing a bogus warning,
> this commit and the code does not make that clear, and that should
> be made clear.
I didn't check the code, but given he said "spurious", I don't think
there's much ambiguity.

Matt

>
> Thanks,
> Rod
>>
>> Modified:
>>   head/usr.sbin/pmc/cmd_pmc_filter.c
>>
>> Modified: head/usr.sbin/pmc/cmd_pmc_filter.c
>> ==
>> --- head/usr.sbin/pmc/cmd_pmc_filter.cMon Jun  4 05:55:40 2018   
>>  (r334608)
>> +++ head/usr.sbin/pmc/cmd_pmc_filter.cMon Jun  4 06:30:35 2018   
>>  (r334609)
>> @@ -145,7 +145,7 @@ pmc_filter_handler(uint32_t *lwplist, int lwpcount, ui
>>   if ((ps = pmclog_open(infd)) == NULL)
>>   errx(EX_OSERR, "ERROR: Cannot allocate pmclog parse state: 
>> %s\n", strerror(errno));
>>
>> - pmccount = 0;
>> + eventcount = pmccount = 0;
>>   while (pmclog_read(ps, ) == 0) {
>>   if (ev.pl_type == PMCLOG_TYPE_INITIALIZE)
>>   memcpy(cpuid, ev.pl_u.pl_i.pl_cpuid, PMC_CPUID_LEN);
>>
>>
>
> --
> Rod Grimes rgri...@freebsd.org
>
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r333466 - in head: contrib/bmake sys/conf sys/kern sys/modules/epoch_test sys/sys sys/tests/epoch

2018-05-11 Thread Matt Joras 
On Fri, May 11, 2018 at 1:13 PM, Bryan Drewery  wrote:
> On 5/10/2018 10:55 AM, Matt Macy wrote:
>> Author: mmacy
>> Date: Thu May 10 17:55:24 2018
>> New Revision: 333466
>> URL: https://svnweb.freebsd.org/changeset/base/333466
>>
>> Log:
>>   Add simple preempt safe epoch API
>>
>>   Read locking is over used in the kernel to guarantee liveness. This API 
>> makes
>>   it easy to provide livenes guarantees without atomics.
>>
>>   Includes epoch_test kernel module to stress test the API.
>>
>>   Documentation will follow initial use case.
>>
>>   Test case and improvements to preemption handling in response to discussion
>>   with mjg@
>>
>>   Reviewed by:imp@, shurd@
>>   Approved by:sbruno@
>>
>> Added:
>>   head/sys/kern/subr_epoch.c   (contents, props changed)
>>   head/sys/modules/epoch_test/
>>   head/sys/modules/epoch_test/Makefile   (contents, props changed)
>>   head/sys/sys/epoch.h   (contents, props changed)
>>   head/sys/tests/epoch/
>>   head/sys/tests/epoch/epoch_test.c   (contents, props changed)
>> Modified:
>>   head/contrib/bmake/job.c
>>   head/sys/conf/files
>>   head/sys/conf/kern.pre.mk
>>   head/sys/kern/kern_malloc.c
>>   head/sys/kern/kern_synch.c
>>   head/sys/kern/subr_trap.c
>>   head/sys/kern/subr_turnstile.c
>>   head/sys/sys/proc.h
>>   head/sys/sys/turnstile.h
>>
>> Modified: head/contrib/bmake/job.c
>> ==
>> --- head/contrib/bmake/job.c  Thu May 10 17:22:04 2018(r333465)
>> +++ head/contrib/bmake/job.c  Thu May 10 17:55:24 2018(r333466)
>> @@ -2121,13 +2121,15 @@ Job_CatchOutput(void)
>>  {
>>  int nready;
>>  Job *job;
>> -int i;
>> +int i, pollToken;
>>
>>  (void)fflush(stdout);
>>
>> + pollToken = 0;
>> +
>>  /* The first fd in the list is the job token pipe */
>>  do {
>> - nready = poll(fds + 1 - wantToken, nfds - 1 + wantToken, POLL_MSEC);
>> + nready = poll(fds + 1 - pollToken, nfds - 1 + pollToken, POLL_MSEC);
>>  } while (nready < 0 && errno == EINTR);
>>
>>  if (nready < 0)
>
> What's up with this?
https://svnweb.freebsd.org/changeset/base/333467
>
> --
> Regards,
> Bryan Drewery
>
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r331727 - in stable/11: share/man/man9 sys/kern sys/sys

2018-03-28 Thread Matt Joras 
Author: mjoras
Date: Thu Mar 29 04:41:45 2018
New Revision: 331727
URL: https://svnweb.freebsd.org/changeset/base/331727

Log:
  MFC r325621, r325622, r331227
  
  Add EVENTHANDLER_LIST and some users.
  
  Also fix a longstanding bug in mtx initialization.

Modified:
  stable/11/share/man/man9/EVENTHANDLER.9
  stable/11/sys/kern/init_main.c
  stable/11/sys/kern/kern_exec.c
  stable/11/sys/kern/kern_exit.c
  stable/11/sys/kern/kern_fork.c
  stable/11/sys/kern/kern_proc.c
  stable/11/sys/kern/kern_thread.c
  stable/11/sys/kern/subr_eventhandler.c
  stable/11/sys/sys/eventhandler.h
Directory Properties:
  stable/11/   (props changed)

Modified: stable/11/share/man/man9/EVENTHANDLER.9
==
--- stable/11/share/man/man9/EVENTHANDLER.9 Thu Mar 29 04:14:37 2018
(r331726)
+++ stable/11/share/man/man9/EVENTHANDLER.9 Thu Mar 29 04:41:45 2018
(r331727)
@@ -23,7 +23,7 @@
 .\" SUCH DAMAGE.
 .\" $FreeBSD$
 .\"
-.Dd October 1, 2017
+.Dd October 31, 2017
 .Dt EVENTHANDLER 9
 .Os
 .Sh NAME
@@ -38,6 +38,9 @@
 .Fn EVENTHANDLER_REGISTER name func arg priority
 .Fn EVENTHANDLER_DEREGISTER name tag
 .Fn EVENTHANDLER_DEREGISTER_NOWAIT name tag
+.Fn EVENTHANDLER_LIST_DECLARE name
+.Fn EVENTHANDLER_LIST_DEFINE name
+.Fn EVENTHANDLER_DIRECT_INVOKE name
 .Ft eventhandler_tag
 .Fo eventhandler_register
 .Fa "struct eventhandler_list *list"
@@ -82,8 +85,13 @@ may be used if the handler does not have a specific pr
 associated with it.
 .Pp
 The normal way to use this subsystem is via the macro interface.
-The macros that can be used for working with event handlers and callback
-function lists are:
+For events that are high frequency it is suggested that you additionally use
+.Fn EVENTHANDLER_LIST_DEFINE
+so that the event handlers can be invoked directly using
+.Fn EVENTHANDLER_DIRECT_INVOKE
+(see below).
+This saves the invoker from having to do a locked traversal of a global
+list of event handler lists.
 .Bl -tag -width indent
 .It Fn EVENTHANDLER_DECLARE
 This macro declares an event handler named by argument
@@ -148,6 +156,27 @@ Additional arguments to the macro after the
 .Fa name
 parameter are passed as the second and subsequent arguments to each
 registered callback function.
+.It Fn EVENTHANDLER_LIST_DEFINE
+This macro defines a reference to an event handler list named by
+argument
+.Fa name .
+It uses
+.Xr SYSINIT 9
+to initialize the reference and the eventhandler list.
+.It Fn EVENTHANDLER_LIST_DECLARE
+This macro declares an event handler list named by argument
+.Fa name .
+This is only needed for users of
+.Fn EVENTHANDLER_DIRECT_INVOKE
+which are not in the same compilation unit of that list's definition.
+.It Fn EVENTHANDLER_DIRECT_INVOKE
+This macro invokes the event handlers registered for the list named by
+argument
+.Fa name .
+This macro can only be used if the list was defined with
+.Fn EVENTHANDLER_LIST_DEFINE .
+The macro is variadic with the same semantics as
+.Fn EVENTHANDLER_INVOKE .
 .El
 .Pp
 The macros are implemented using the following functions:
@@ -315,7 +344,7 @@ This is never called.
 .It Vt process_fork
 Callbacks invoked when a process forks a child.
 .It Vt process_init
-Callback invoked when a process is initalized.
+Callback invoked when a process is initialized.
 .It Vt random_adaptor_attach
 Callback invoked when a new random module has been loaded.
 .It Vt register_framebuffer
@@ -337,7 +366,7 @@ Callback invoked when a thread object is created.
 .It Vt thread_dtor
 Callback invoked when a thread object is destroyed.
 .It Vt thread_init
-Callback invoked when a thread object is initalized.
+Callback invoked when a thread object is initialized.
 .It Vt thread_fini
 Callback invoked when a thread object is deinitalized.
 .It Vt usb_dev_configured
@@ -384,4 +413,6 @@ facility first appeared in
 .Fx 4.0 .
 .Sh AUTHORS
 This manual page was written by
-.An Joseph Koshy Aq Mt jko...@freebsd.org .
+.An Joseph Koshy Aq Mt jko...@freebsd.org
+and
+.An Matt Joras Aq Mt mjo...@freebsd.org .

Modified: stable/11/sys/kern/init_main.c
==
--- stable/11/sys/kern/init_main.c  Thu Mar 29 04:14:37 2018
(r331726)
+++ stable/11/sys/kern/init_main.c  Thu Mar 29 04:41:45 2018
(r331727)
@@ -136,6 +136,11 @@ SET_DECLARE(sysinit_set, struct sysinit);
 struct sysinit **sysinit, **sysinit_end;
 struct sysinit **newsysinit, **newsysinit_end;
 
+EVENTHANDLER_LIST_DECLARE(process_init);
+EVENTHANDLER_LIST_DECLARE(thread_init);
+EVENTHANDLER_LIST_DECLARE(process_ctor);
+EVENTHANDLER_LIST_DECLARE(thread_ctor);
+
 /*
  * Merge a new sysinit set into the current set, reallocating it if
  * necessary.  This can only be called after malloc is running.
@@ -585,10 +590,10 @@ proc0_init(void *dummy __unused)
 * Call the init and ctor for the new thread and proc.  We wait
 * to do this u

svn commit: r331227 - head/sys/kern

2018-03-19 Thread Matt Joras 
Author: mjoras
Date: Mon Mar 19 22:43:27 2018
New Revision: 331227
URL: https://svnweb.freebsd.org/changeset/base/331227

Log:
  Fix initialization of eventhandler mutex.
  
  mtx_init does not do a copy of the name string it is passed. The
  eventhandler code incorrectly passed the parameter string directly to
  mtx_init instead of using the copy it makes. This was an existing
  problem with the code that I dutifully copied over in my changes in r325621.
  
  Reported by:  Anton Rang 
  Reviewed by:  rstone, markj
  Approved by:  rstone (mentor)
  MFC after:1 week
  Sponsored by: Dell EMC Isilon
  Differential Revision:https://reviews.freebsd.org/D14764

Modified:
  head/sys/kern/subr_eventhandler.c

Modified: head/sys/kern/subr_eventhandler.c
==
--- head/sys/kern/subr_eventhandler.c   Mon Mar 19 21:26:32 2018
(r331226)
+++ head/sys/kern/subr_eventhandler.c   Mon Mar 19 22:43:27 2018
(r331227)
@@ -90,9 +90,10 @@ eventhandler_find_or_create_list(const char *name)
CTR2(KTR_EVH, "%s: creating list \"%s\"", __func__, name);
list = new_list;
TAILQ_INIT(>el_entries);
-   mtx_init(>el_lock, name, "eventhandler list", MTX_DEF);
list->el_name = (char *)(list + 1);
strcpy(list->el_name, name);
+   mtx_init(>el_lock, list->el_name, "eventhandler list",
+   MTX_DEF);
TAILQ_INSERT_HEAD(_lists, list, el_link);
}
}
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r328430 - head/sbin/devd

2018-01-25 Thread Matt Joras 
On Thu, Jan 25, 2018 at 8:52 PM, Warner Losh  wrote:
>
>
> On Thu, Jan 25, 2018 at 9:40 PM, Eitan Adler  wrote:
>>
>> Author: eadler
>> Date: Fri Jan 26 04:40:41 2018
>> New Revision: 328430
>> URL: https://svnweb.freebsd.org/changeset/base/328430
>>
>> Log:
>>   devd: minor nits
>>
>>   - mark usage as noreturn
>>   - config does not need a virtual destructor
>
>
> Everything needs a virtual destructor...  Please back that part of this
> out...
>
> Warner
Needs? If there's not inheritance there's no _need_ to do it, and it's
arguably superfluous. That being said it's also an arguably
superfluous change to remove it.

Matt
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r327354 - head/sys/vm

2018-01-18 Thread Matt Joras 
On Thu, Jan 18, 2018 at 1:04 PM, Warner Losh  wrote:
>
>> 1. Allowing local / block scoped variables
>> 2. Allowing C99 for loop initial declarations
>>
>> But I could see the argument that (2) is just a boring subset of (1).
>
>
> Tell me again where #1/#2 are disallowed? I can't seem to find that in
> style(9), except by a weak example of there being no examples of #1 or #2.
>
> Warner
>
It is sort of implied by the sections stating the rules for declaring variables:
"When declaring variables in functions declare them sorted by size,
then n alphabetical order; multiple ones per line are okay."
And this:
"/* Insert an empty line if the function has no local variables. */"

I don't see a more explicit note about them.

Matt
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r327559 - in head: . sys/net

2018-01-05 Thread Matt Joras 
On Fri, Jan 5, 2018 at 9:32 AM, Eugene Grosbein <eu...@grosbein.net> wrote:
> 06.01.2018 0:28, Matt Joras wrote:
>
>> For what it's worth, this was the conclusion I came to, and at Isilon
>> we've made the same change being discussed here. For the case of
>> drivers that end up using a queue index for the flowid, you end up
>> with pathological behavior on the lagg; the flowid ends up getting
>> right shifted by (default) 16. So in the case of e.g. two bxe(4)
>> interfaces with 4 queues, you always end up choosing the interface in
>> the lagg at index 0.
>
> Not all drivers have this bug. These are drivers that needs to be fixed to 
> not shift by 16, not lagg.
>
I don't follow. It is if_lagg that does the shifting. For loadbalance
it is done directly in lagg_snd_tag_alloc, and for LACP it is done in
a separate fucntion, lacp_select_tx_port_by_hash. For both it shifts
the flowid by the flowid_shift set on the lagg sc, which defaults to
16.

You could make the argument that we should fix every driver that sets
a queue index to instead use an RSS hash, but that seems like more
work than simply disabling the use of flowid in if_lagg by default.
For cases where this has an appreciable impact on forwarding
performance the sysctl can be flipped back. That seems more reasonable
to me than making laggs effectively useless for anyone using any one
of a random set of drivers that set the flowid to a queue index (grep
for "flowid =" and you can see which drivers do this).

Matt
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r327559 - in head: . sys/net

2018-01-05 Thread Matt Joras 
On Fri, Jan 5, 2018 at 7:42 AM, Steven Hartland  wrote:
> My current thinking is that flowid shouldn't be used for either LACP or
> loadbalance protocols as doing so will almost certainly lead to unexpected
> behavior (the stated lagghash may not be valid).
>
> Regards
> Steve
>

For what it's worth, this was the conclusion I came to, and at Isilon
we've made the same change being discussed here. For the case of
drivers that end up using a queue index for the flowid, you end up
with pathological behavior on the lagg; the flowid ends up getting
right shifted by (default) 16. So in the case of e.g. two bxe(4)
interfaces with 4 queues, you always end up choosing the interface in
the lagg at index 0.

This was enough to make us completely disable it. Customers started
noticing when they upgraded from the BSD7-based release (which
predates flowid) to the BSD10-based release. It is my impression that
this sort of situation is probably more common in FreeBSD, especially
since some drivers only set the flowid to an RSS hash when RSS is
defined, otherwise defaulting to a queue index.

Matt
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r326554 - in head: . usr.bin/sponge usr.bin/sponge/tests usr.bin/tee

2017-12-05 Thread Matt Joras 
On Tue, Dec 5, 2017 at 8:06 AM, Devin Teske  wrote:
>
>
> The problems I have are:
>
> 1. Should be in ports
>
> Not pre-installed on Linux, why should we have it in base?
"Pre-installed on Linux" is meaningless. The closest analog to our
base is arguably GNU coreutils. It is indeed not part of GNU coreutils
but then again there are several things in our base that are not in
coreutils, and vice versa. Why should we have anything in base? If
people find it useful and it doens't have a high cost of
maintainership then why not have it?

> If in base, people will target it thinking it solves a need that can't
> otherwise be solved and thus end up creating a script that is less portable
> because it is encumbered with dependencies specific to our base.
It's not even a homegrown idea though... As we've already covered this
is a tool that exists in the broader OSS ecosystem. As long as it is
compatible with the more common implementation I don't see the issue.
Anything one writes using it is just "encumbered" with a dependency on
sponge.

> 2. Teaches bad practice
>
> sed ... somefile | sponge somefile
>
> Ignores if there is a sed error and indiscriminately zeroes somefile.

Calling this unequivocally bad practice is silly. There are plenty
uses of sponge that aren't bad practice. I have a git commit hook that
utilizes sponge to do the same "auto-culling" that our svn patches do.
I like the sponge version better than creating temporary files myself:

sed '/^$/d' $(git config commit.message) | awk 'NR==FNR{a[$0];next}
!($0 in a)' /dev/fd/0 "$1" | sponge "$1"

> 3. Solution in search of a problem
Again, stating this unequivocally is silly. I discovered sponge years
ago when I was searching how best to handle something where I wanted
to write output back to the same file in a shell pipeline. I was
literally someone with a problem in search of a solution, and that
solution was and still is sponge. Since then I have seen it
recommended numerous times in passing.

Matt
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r326554 - in head: . usr.bin/sponge usr.bin/sponge/tests usr.bin/tee

2017-12-05 Thread Matt Joras 
On Dec 5, 2017 7:35 AM, "Devin Teske"  wrote:


> On Dec 5, 2017, at 5:00 AM, Hans Petter Selasky  wrote:
>
>> On 12/05/17 13:58, Rodney W. Grimes wrote:
>> Further more, why does freebsd need this in base?
>
> Hi,
>
> I think this is useful. It could replace the "-i " (intermediate) option
for "sed" for example. It avoids creating temporary files when filtering
files, right?
>
> --HPS
>

Wth is wrong with:

data=$( sed -e '...' somefile ) &&
echo "$data" > somefile

or

set -e
data=...
echo "$data" > ...

or

exec 3< ... <&3

or

(I digress)

Infinite variations, but the gist is that sponge looks to be trying to help
sh(1)/similar when help is unneeded.

Why buffer data into memory via fork-exec-pipe to sponge when you can
buffer to native namespace without pipe to sponge?

Am I missing something? Why do we need sponge(1)?
--
Devin

I do believe you are sort of missing the point. It is a utility that is
explicitly useful in shell pipelines, so when you want to do things as
one-liners. I like the utility and use the one from ports and my own
version in various things here and there. It is a common utility installed
in Linux distros and the top answer on Google for questions such as
"redirect shell output to same file". I think the outrage about adding a
tiny utility that's common elsewhere is a bit silly.

As for the implementation, I have my own version of sponge (hobby program
written in rust so not base-worthy), and it uses explicit temporary files
for larger outputs.

Matt
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r325728 - head/lib/libkvm

2017-11-11 Thread Matt Joras 
On 11/11/2017 15:30, Will Andrews wrote:
> Author: will
> Date: Sat Nov 11 23:30:58 2017
> New Revision: 325728
> URL: https://svnweb.freebsd.org/changeset/base/325728
>
> Log:
>   libkvm: add kvm_walk_pages API.
>   
>   This API allows callers to enumerate all known pages, including any
>   direct map & kernel map virtual addresses, physical addresses, size,
>   offset into the core, & protection configured.
>   
>   For architectures that support direct map addresses, also generate pages
>   for any direct map only addresses that are not associated with kernel
>   map addresses.
>   
>   Fix page size portability issue left behind from previous kvm page table
>   lookup interface.
>   
>   Reviewed by:jhb
>   Sponsored by:   Backtrace I/O
>   Differential Revision:  https://reviews.freebsd.org/D12279

This broke powerpc, riscv64, sparc64:

*23:36:15* /usr/src/lib/libkvm/kvm_private.c: In function '_kvm_bitmap_init':
*23:36:15* /usr/src/lib/libkvm/kvm_private.c:702: warning: declaration of 
'index' shadows a global declaration
*23:36:15* /usr/obj/usr/src/powerpc.powerpc/tmp/usr/include/strings.h:60: 
warning: shadowed declaration is here
*23:36:15* /usr/src/lib/libkvm/kvm_private.c: In function '_kvm_bitmap_next':
*23:36:15* /usr/src/lib/libkvm/kvm_private.c:723: warning: declaration of 
'index' shadows a global declaration
*23:36:15* /usr/obj/usr/src/powerpc.powerpc/tmp/usr/include/strings.h:60: 
warning: shadowed declaration is here
*23:36:15* *** [kvm_private.o] Error code 1

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r325622 - head/share/man/man9

2017-11-09 Thread Matt Joras 
Author: mjoras
Date: Thu Nov  9 23:36:10 2017
New Revision: 325622
URL: https://svnweb.freebsd.org/changeset/base/325622

Log:
  Correct mistake in manpage.
  
  Reported by:  pluknet
  Approved by:  rstone (mentor)
  MFC with: r325621
  Pointy hat to:  mjoras

Modified:
  head/share/man/man9/EVENTHANDLER.9

Modified: head/share/man/man9/EVENTHANDLER.9
==
--- head/share/man/man9/EVENTHANDLER.9  Thu Nov  9 22:51:48 2017
(r325621)
+++ head/share/man/man9/EVENTHANDLER.9  Thu Nov  9 23:36:10 2017
(r325622)
@@ -86,7 +86,7 @@ associated with it.
 .Pp
 The normal way to use this subsystem is via the macro interface.
 For events that are high frequency it is suggested that you additionally use
-.Fn EVENTHANDLER_DEFINE_LIST
+.Fn EVENTHANDLER_LIST_DEFINE
 so that the event handlers can be invoked directly using
 .Fn EVENTHANDLER_DIRECT_INVOKE
 (see below).
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r325621 - in head: share/man/man9 sys/kern sys/sys

2017-11-09 Thread Matt Joras 
 is deinitalized.
 .It Vt usb_dev_configured
@@ -384,4 +413,6 @@ facility first appeared in
 .Fx 4.0 .
 .Sh AUTHORS
 This manual page was written by
-.An Joseph Koshy Aq Mt jko...@freebsd.org .
+.An Joseph Koshy Aq Mt jko...@freebsd.org
+and
+.An Matt Joras Aq Mt mjo...@freebsd.org .

Modified: head/sys/kern/init_main.c
==
--- head/sys/kern/init_main.c   Thu Nov  9 22:26:49 2017(r325620)
+++ head/sys/kern/init_main.c   Thu Nov  9 22:51:48 2017(r325621)
@@ -136,6 +136,11 @@ SET_DECLARE(sysinit_set, struct sysinit);
 struct sysinit **sysinit, **sysinit_end;
 struct sysinit **newsysinit, **newsysinit_end;
 
+EVENTHANDLER_LIST_DECLARE(process_init);
+EVENTHANDLER_LIST_DECLARE(thread_init);
+EVENTHANDLER_LIST_DECLARE(process_ctor);
+EVENTHANDLER_LIST_DECLARE(thread_ctor);
+
 /*
  * Merge a new sysinit set into the current set, reallocating it if
  * necessary.  This can only be called after malloc is running.
@@ -580,10 +585,10 @@ proc0_init(void *dummy __unused)
 * Call the init and ctor for the new thread and proc.  We wait
 * to do this until all other structures are fairly sane.
 */
-   EVENTHANDLER_INVOKE(process_init, p);
-   EVENTHANDLER_INVOKE(thread_init, td);
-   EVENTHANDLER_INVOKE(process_ctor, p);
-   EVENTHANDLER_INVOKE(thread_ctor, td);
+   EVENTHANDLER_DIRECT_INVOKE(process_init, p);
+   EVENTHANDLER_DIRECT_INVOKE(thread_init, td);
+   EVENTHANDLER_DIRECT_INVOKE(process_ctor, p);
+   EVENTHANDLER_DIRECT_INVOKE(thread_ctor, td);
 
/*
 * Charge root for one process.

Modified: head/sys/kern/kern_exec.c
==
--- head/sys/kern/kern_exec.c   Thu Nov  9 22:26:49 2017(r325620)
+++ head/sys/kern/kern_exec.c   Thu Nov  9 22:51:48 2017(r325621)
@@ -144,6 +144,8 @@ static int map_at_zero = 0;
 SYSCTL_INT(_security_bsd, OID_AUTO, map_at_zero, CTLFLAG_RWTUN, _at_zero, 
0,
 "Permit processes to map an object at virtual address 0.");
 
+EVENTHANDLER_LIST_DECLARE(process_exec);
+
 static int
 sysctl_kern_ps_strings(SYSCTL_HANDLER_ARGS)
 {
@@ -1071,7 +1073,7 @@ exec_new_vmspace(struct image_params *imgp, struct sys
imgp->sysent = sv;
 
/* May be called with Giant held */
-   EVENTHANDLER_INVOKE(process_exec, p, imgp);
+   EVENTHANDLER_DIRECT_INVOKE(process_exec, p, imgp);
 
/*
 * Blow away entire process VM, if address space not shared,

Modified: head/sys/kern/kern_exit.c
==
--- head/sys/kern/kern_exit.c   Thu Nov  9 22:26:49 2017(r325620)
+++ head/sys/kern/kern_exit.c   Thu Nov  9 22:51:48 2017(r325621)
@@ -99,6 +99,8 @@ SDT_PROBE_DEFINE1(proc, , , exit, "int");
 /* Hook for NFS teardown procedure. */
 void (*nlminfo_release_p)(struct proc *p);
 
+EVENTHANDLER_LIST_DECLARE(process_exit);
+
 struct proc *
 proc_realparent(struct proc *child)
 {
@@ -329,7 +331,7 @@ exit1(struct thread *td, int rval, int signo)
 * Event handler could change exit status.
 * XXX what if one of these generates an error?
 */
-   EVENTHANDLER_INVOKE(process_exit, p);
+   EVENTHANDLER_DIRECT_INVOKE(process_exit, p);
 
/*
 * If parent is waiting for us to exit or exec,

Modified: head/sys/kern/kern_fork.c
==
--- head/sys/kern/kern_fork.c   Thu Nov  9 22:26:49 2017(r325620)
+++ head/sys/kern/kern_fork.c   Thu Nov  9 22:51:48 2017(r325621)
@@ -97,6 +97,8 @@ struct fork_args {
 };
 #endif
 
+EVENTHANDLER_LIST_DECLARE(process_fork);
+
 /* ARGSUSED */
 int
 sys_fork(struct thread *td, struct fork_args *uap)
@@ -699,7 +701,7 @@ do_fork(struct thread *td, struct fork_req *fr, struct
 * Both processes are set up, now check if any loadable modules want
 * to adjust anything.
 */
-   EVENTHANDLER_INVOKE(process_fork, p1, p2, fr->fr_flags);
+   EVENTHANDLER_DIRECT_INVOKE(process_fork, p1, p2, fr->fr_flags);
 
/*
 * Set the child start time and mark the process as being complete.

Modified: head/sys/kern/kern_proc.c
==
--- head/sys/kern/kern_proc.c   Thu Nov  9 22:26:49 2017(r325620)
+++ head/sys/kern/kern_proc.c   Thu Nov  9 22:51:48 2017(r325621)
@@ -151,6 +151,17 @@ const int thread_off_td_oncpu = offsetof(struct thread
 const int thread_off_td_pcb = offsetof(struct thread, td_pcb);
 const int thread_off_td_plist = offsetof(struct thread, td_plist);
 
+EVENTHANDLER_LIST_DEFINE(process_ctor);
+EVENTHANDLER_LIST_DEFINE(process_dtor);
+EVENTHANDLER_LIST_DEFINE(process_init);
+EVENTHANDLER_LIST_DEFINE(process_fini);
+EVENTHANDLER_LIST_DEFINE(process_exit);
+EVENTHANDLE

svn commit: r324921 - head/sys/fs/tmpfs

2017-10-23 Thread Matt Joras 
Author: mjoras
Date: Mon Oct 23 15:43:38 2017
New Revision: 324921
URL: https://svnweb.freebsd.org/changeset/base/324921

Log:
  Move clear_unrhdr to tmpfs_free_tmp.
  
  Clearing the unr in tmpfs_unmount is not correct. In the case of
  multiple references to the tmpfs mount (e.g. when there are lookup
  threads using it) it will not be the one to finish tmpfs_free_tmp. In
  those cases tmpfs_free_node_locked will be the final one to execute
  tmpfs_free_tmp, and until then the unr must be valid.
  
  Reported by:  pho
  Approved/reviewed by: rstone (mentor)
  Sponsored by: Dell EMC Isilon
  Differential Revision:https://reviews.freebsd.org/D12749

Modified:
  head/sys/fs/tmpfs/tmpfs_vfsops.c

Modified: head/sys/fs/tmpfs/tmpfs_vfsops.c
==
--- head/sys/fs/tmpfs/tmpfs_vfsops.cMon Oct 23 15:34:05 2017
(r324920)
+++ head/sys/fs/tmpfs/tmpfs_vfsops.cMon Oct 23 15:43:38 2017
(r324921)
@@ -317,8 +317,6 @@ tmpfs_unmount(struct mount *mp, int mntflags)
TMPFS_NODE_UNLOCK(node);
}
 
-   clear_unrhdr(tmp->tm_ino_unr);
-
mp->mnt_data = NULL;
tmpfs_free_tmp(tmp);
vfs_write_resume(mp, VR_START_WRITE);
@@ -344,6 +342,7 @@ tmpfs_free_tmp(struct tmpfs_mount *tmp)
 
uma_zdestroy(tmp->tm_dirent_pool);
uma_zdestroy(tmp->tm_node_pool);
+   clear_unrhdr(tmp->tm_ino_unr);
delete_unrhdr(tmp->tm_ino_unr);
 
mtx_destroy(>tm_allnode_lock);
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r324666 - head/sys/kern

2017-10-16 Thread Matt Joras 
On 10/16/2017 09:14, Matt Joras wrote:
> Author: mjoras
> Date: Mon Oct 16 16:14:50 2017
> New Revision: 324666
> URL: https://svnweb.freebsd.org/changeset/base/324666
>
> Log:
>   Properly reset the fields in clean_unrhdr.
>   
>   In r324542 I neglected to reset the first and last fields of struct
>   unrhdr. This causes a tmpfs to fail the unr(9) consistency checks with
>   DIAGNOSTIC on. Fix this by resetting the fields by calling init_unrhdr.
>   While here, change a loop to use TAILQ_FOREACH_SAFE since it is more
>   readable and equally fast.
>   
>   Reported by:David Wolfskill <da...@catwhisker.org>
>   Approved by:rstone (mentor)
>   Sponsored by:   Dell EMC Isilon
>
Differential Revision: https://reviews.freebsd.org/D12662

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r324666 - head/sys/kern

2017-10-16 Thread Matt Joras 
Author: mjoras
Date: Mon Oct 16 16:14:50 2017
New Revision: 324666
URL: https://svnweb.freebsd.org/changeset/base/324666

Log:
  Properly reset the fields in clean_unrhdr.
  
  In r324542 I neglected to reset the first and last fields of struct
  unrhdr. This causes a tmpfs to fail the unr(9) consistency checks with
  DIAGNOSTIC on. Fix this by resetting the fields by calling init_unrhdr.
  While here, change a loop to use TAILQ_FOREACH_SAFE since it is more
  readable and equally fast.
  
  Reported by:  David Wolfskill 
  Approved by:  rstone (mentor)
  Sponsored by: Dell EMC Isilon

Modified:
  head/sys/kern/subr_unit.c

Modified: head/sys/kern/subr_unit.c
==
--- head/sys/kern/subr_unit.c   Mon Oct 16 15:16:24 2017(r324665)
+++ head/sys/kern/subr_unit.c   Mon Oct 16 16:14:50 2017(r324666)
@@ -373,18 +373,17 @@ clear_unrhdr(struct unrhdr *uh)
 
KASSERT(TAILQ_EMPTY(>ppfree),
("unrhdr has postponed item for free"));
-   up = TAILQ_FIRST(>head);
-   while (up != NULL) {
-   uq = TAILQ_NEXT(up, list);
+   TAILQ_FOREACH_SAFE(up, >head, list, uq) {
if (up->ptr != uh) {
Free(up->ptr);
}
Free(up);
-   up = uq;
}
-   TAILQ_INIT(>head);
uh->busy = 0;
uh->alloc = 0;
+   init_unrhdr(uh, uh->low, uh->high, uh->mtx);
+
+   check_unrhdr(uh, __LINE__);
 }
 
 static __inline int
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r324541 - in head: share/man/man9 sys/kern sys/sys

2017-10-14 Thread Matt Joras 
On 10/14/2017 11:03, Mark Johnston wrote:
> TAILQ_FOREACH_SAFE just fetches the next element at the beginning of
> each loop iteration rather than at the end, same as the current
> implementation of clear_unrhdr() does. There's no change to the code
> generated by clang when I replace your loop with:
>
>   TAILQ_FOREACH_SAFE(up, >head, list, uq) {
>   if (up->ptr != uh) {
>   Free(up->ptr);
>   }
>   Free(up);
>   }

Ah, that's a fair point... I was assuming Ngie was suggesting doing a
TAILQ_REMOVE during each iteration, since that's probably the typical
reason to do TAILQ_FOREACH_SAFE. That loop looks better. It would
probably be good to change the queue(3) manpage to suggest that for
deletion.

Matt

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r324541 - in head: share/man/man9 sys/kern sys/sys

2017-10-14 Thread Matt Joras 
On 10/13/2017 22:12, Ngie Cooper (yaneurabeya) wrote:
>> Modified: head/sys/kern/subr_unit.c
>> ==
>> --- head/sys/kern/subr_unit.cWed Oct 11 20:36:22 2017
>> (r324540)
>> +++ head/sys/kern/subr_unit.cWed Oct 11 21:53:50 2017
>> (r324541)
>> @@ -366,6 +366,27 @@ delete_unrhdr(struct unrhdr *uh)
>>  Free(uh);
>> }
>>
>> +void
>> +clear_unrhdr(struct unrhdr *uh)
>> +{
>> +struct unr *up, *uq;
>> +
>> +KASSERT(TAILQ_EMPTY(>ppfree),
>> +("unrhdr has postponed item for free"));
>> +up = TAILQ_FIRST(>head);
>> +while (up != NULL) {
> Could this be done with TAILQ_FOREACH_SAFE?
> -Ngie


Yes but it is arguably inferior to do so. This while loop is
theoretically faster since there is no need to individually remove the
elements when you intend to delete every element.

Matt

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r324542 - head/sys/fs/tmpfs

2017-10-11 Thread Matt Joras 
Author: mjoras
Date: Wed Oct 11 21:53:53 2017
New Revision: 324542
URL: https://svnweb.freebsd.org/changeset/base/324542

Log:
  When unmounting a tmpfs, do not call free_unr.
  
  tmpfs uses unr(9) to allocate inodes. Previously when unmounting it
  would individually free the units when it freed each vnode. This is
  unnecessary as we can use the newly-added unrhdr_clear function to clear
  out the unr in onde go. This measurably reduces the time to unmount a
  tmpfs with many files.
  
  Reviewed by:  cem, lidl
  Approved by:  rstone (mentor)
  Sponsored by: Dell EMC Isilon
  Differential Revision:https://reviews.freebsd.org/D12591

Modified:
  head/sys/fs/tmpfs/tmpfs_subr.c
  head/sys/fs/tmpfs/tmpfs_vfsops.c

Modified: head/sys/fs/tmpfs/tmpfs_subr.c
==
--- head/sys/fs/tmpfs/tmpfs_subr.c  Wed Oct 11 21:53:50 2017
(r324541)
+++ head/sys/fs/tmpfs/tmpfs_subr.c  Wed Oct 11 21:53:53 2017
(r324542)
@@ -362,7 +362,13 @@ tmpfs_free_node_locked(struct tmpfs_mount *tmp, struct
panic("tmpfs_free_node: type %p %d", node, (int)node->tn_type);
}
 
-   free_unr(tmp->tm_ino_unr, node->tn_id);
+   /*
+* If we are unmounting there is no need for going through the overhead
+* of freeing the inodes from the unr individually, so free them all in
+* one go later.
+*/
+   if (!detach)
+   free_unr(tmp->tm_ino_unr, node->tn_id);
uma_zfree(tmp->tm_node_pool, node);
TMPFS_LOCK(tmp);
tmpfs_free_tmp(tmp);

Modified: head/sys/fs/tmpfs/tmpfs_vfsops.c
==
--- head/sys/fs/tmpfs/tmpfs_vfsops.cWed Oct 11 21:53:50 2017
(r324541)
+++ head/sys/fs/tmpfs/tmpfs_vfsops.cWed Oct 11 21:53:53 2017
(r324542)
@@ -317,6 +317,8 @@ tmpfs_unmount(struct mount *mp, int mntflags)
TMPFS_NODE_UNLOCK(node);
}
 
+   clear_unrhdr(tmp->tm_ino_unr);
+
mp->mnt_data = NULL;
tmpfs_free_tmp(tmp);
vfs_write_resume(mp, VR_START_WRITE);
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r324541 - in head: share/man/man9 sys/kern sys/sys

2017-10-11 Thread Matt Joras 
Author: mjoras
Date: Wed Oct 11 21:53:50 2017
New Revision: 324541
URL: https://svnweb.freebsd.org/changeset/base/324541

Log:
  Add clearing function for unr(9).
  
  Previously before you could call unrhdr_delete you needed to
  individually free every allocated unit. It is useful to be able to tear
  down the unr without having to go through this process, as it is
  significantly faster than freeing the individual units.
  
  Reviewed by:  cem, lidl
  Approved by:  rstone (mentor)
  Sponsored by: Dell EMC Isilon
  Differential Revision:https://reviews.freebsd.org/D12591

Modified:
  head/share/man/man9/Makefile
  head/share/man/man9/unr.9
  head/sys/kern/subr_unit.c
  head/sys/sys/systm.h

Modified: head/share/man/man9/Makefile
==
--- head/share/man/man9/MakefileWed Oct 11 20:36:22 2017
(r324540)
+++ head/share/man/man9/MakefileWed Oct 11 21:53:50 2017
(r324541)
@@ -414,6 +414,7 @@ MAN=accept_filter.9 \
 MLINKS=unr.9 alloc_unr.9 \
unr.9 alloc_unrl.9 \
unr.9 alloc_unr_specific.9 \
+   unr.9 clear_unrhdr.9 \
unr.9 delete_unrhdr.9 \
unr.9 free_unr.9 \
unr.9 new_unrhdr.9

Modified: head/share/man/man9/unr.9
==
--- head/share/man/man9/unr.9   Wed Oct 11 20:36:22 2017(r324540)
+++ head/share/man/man9/unr.9   Wed Oct 11 21:53:50 2017(r324541)
@@ -24,11 +24,12 @@
 .\"
 .\" $FreeBSD$
 .\"
-.Dd July 5, 2010
+.Dd October 4, 2017
 .Dt UNR 9
 .Os
 .Sh NAME
 .Nm new_unrhdr ,
+.Nm clear_unrhdr ,
 .Nm delete_unrhdr ,
 .Nm alloc_unr ,
 .Nm alloc_unr_specific ,
@@ -39,6 +40,8 @@
 .Ft "struct unrhdr *"
 .Fn new_unrhdr "int low" "int high" "struct mtx *mutex"
 .Ft void
+.Fn clear_unrhdr "struct unrhdr *uh"
+.Ft void
 .Fn delete_unrhdr "struct unrhdr *uh"
 .Ft int
 .Fn alloc_unr "struct unrhdr *uh"
@@ -70,8 +73,16 @@ is not
 .Dv NULL ,
 it is used for locking when allocating and freeing units.
 Otherwise, internal mutex is used.
+.It Fn clear_unrhdr uh
+Clear all units from the specified unit number allocator entity.
+This function resets the entity as if it were just initialized with
+.Fn new_unrhdr .
 .It Fn delete_unrhdr uh
-Destroy specified unit number allocator entity.
+Delete specified unit number allocator entity.
+This function frees the memory associated with the entity, it does not free
+any units.
+To free all units use
+.Fn clear_unrhdr .
 .It Fn alloc_unr uh
 Return a new unit number.
 The lowest free number is always allocated.

Modified: head/sys/kern/subr_unit.c
==
--- head/sys/kern/subr_unit.c   Wed Oct 11 20:36:22 2017(r324540)
+++ head/sys/kern/subr_unit.c   Wed Oct 11 21:53:50 2017(r324541)
@@ -366,6 +366,27 @@ delete_unrhdr(struct unrhdr *uh)
Free(uh);
 }
 
+void
+clear_unrhdr(struct unrhdr *uh)
+{
+   struct unr *up, *uq;
+
+   KASSERT(TAILQ_EMPTY(>ppfree),
+   ("unrhdr has postponed item for free"));
+   up = TAILQ_FIRST(>head);
+   while (up != NULL) {
+   uq = TAILQ_NEXT(up, list);
+   if (up->ptr != uh) {
+   Free(up->ptr);
+   }
+   Free(up);
+   up = uq;
+   }
+   TAILQ_INIT(>head);
+   uh->busy = 0;
+   uh->alloc = 0;
+}
+
 static __inline int
 is_bitmap(struct unrhdr *uh, struct unr *up)
 {

Modified: head/sys/sys/systm.h
==
--- head/sys/sys/systm.hWed Oct 11 20:36:22 2017(r324540)
+++ head/sys/sys/systm.hWed Oct 11 21:53:50 2017(r324541)
@@ -450,6 +450,7 @@ struct unrhdr;
 struct unrhdr *new_unrhdr(int low, int high, struct mtx *mutex);
 void init_unrhdr(struct unrhdr *uh, int low, int high, struct mtx *mutex);
 void delete_unrhdr(struct unrhdr *uh);
+void clear_unrhdr(struct unrhdr *uh);
 void clean_unrhdr(struct unrhdr *uh);
 void clean_unrhdrl(struct unrhdr *uh);
 int alloc_unr(struct unrhdr *uh);
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r323633 - stable/11/sys/net

2017-09-15 Thread Matt Joras 
Author: mjoras
Date: Sat Sep 16 02:10:36 2017
New Revision: 323633
URL: https://svnweb.freebsd.org/changeset/base/323633

Log:
  MFC r323513:
  Allow vlan interfaces to rx through netmap(4).

Modified:
  stable/11/sys/net/if_vlan.c
Directory Properties:
  stable/11/   (props changed)

Modified: stable/11/sys/net/if_vlan.c
==
--- stable/11/sys/net/if_vlan.c Fri Sep 15 22:56:39 2017(r323632)
+++ stable/11/sys/net/if_vlan.c Sat Sep 16 02:10:36 2017(r323633)
@@ -1376,7 +1376,7 @@ vlan_input(struct ifnet *ifp, struct mbuf *m)
VLAN_RUNLOCK();
 
/* Pass it back through the parent's input routine. */
-   (*ifp->if_input)(ifv->ifv_ifp, m);
+   (*ifv->ifv_ifp->if_input)(ifv->ifv_ifp, m);
 }
 
 static void
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r323513 - head/sys/net

2017-09-12 Thread Matt Joras 
Author: mjoras
Date: Wed Sep 13 00:25:09 2017
New Revision: 323513
URL: https://svnweb.freebsd.org/changeset/base/323513

Log:
  Allow vlan interfaces to rx through netmap(4).
  
  Normally after receiving a packet, a vlan(4) interface sends the packet
  back through its parent interface's rx routine so that it can be
  processed as an untagged frame. It does this by using the parent's
  ifp->if_input. This is incompatible with netmap(4), which replaces the
  vlan(4) interface's if_input with a netmap(4) hook. Fix this by using
  the vlan(4) interface's ifp instead of the parent's directly.
  
  Reported by:  Harry Schmalzbauer 
  Reviewed by:  rstone
  Approved by:  rstone (mentor)
  MFC after:3 days
  Sponsored by: Dell EMC Isilon
  Differential Revision:https://reviews.freebsd.org/D12191

Modified:
  head/sys/net/if_vlan.c

Modified: head/sys/net/if_vlan.c
==
--- head/sys/net/if_vlan.c  Tue Sep 12 23:58:38 2017(r323512)
+++ head/sys/net/if_vlan.c  Wed Sep 13 00:25:09 2017(r323513)
@@ -1384,7 +1384,7 @@ vlan_input(struct ifnet *ifp, struct mbuf *m)
VLAN_RUNLOCK();
 
/* Pass it back through the parent's input routine. */
-   (*ifp->if_input)(ifv->ifv_ifp, m);
+   (*ifv->ifv_ifp->if_input)(ifv->ifv_ifp, m);
 }
 
 static void
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r323477 - stable/11/sys/net

2017-09-11 Thread Matt Joras 
Author: mjoras
Date: Tue Sep 12 03:54:04 2017
New Revision: 323477
URL: https://svnweb.freebsd.org/changeset/base/323477

Log:
  MFC r322548: Rework vlan(4) locking.
  
  Previously the locking of vlan(4) interfaces was not very comprehensive.
  Particularly there was very little protection against the destruction of
  active vlan(4) interfaces or concurrent modification of a vlan(4)
  interface. The former readily produced several different panics.
  
  The changes can be summarized as using two global vlan locks (an
  rmlock(9) and an sx(9)) to protect accesses to the if_vlantrunk field of
  struct ifnet, in addition to other places where global exclusive access
  is required. vlan(4) should now be much more resilient to the destruction
  of active interfaces and concurrent calls into the configuration path.

Modified:
  stable/11/sys/net/if_vlan.c
Directory Properties:
  stable/11/   (props changed)

Modified: stable/11/sys/net/if_vlan.c
==
--- stable/11/sys/net/if_vlan.c Tue Sep 12 00:44:16 2017(r323476)
+++ stable/11/sys/net/if_vlan.c Tue Sep 12 03:54:04 2017(r323477)
@@ -1,6 +1,7 @@
 /*-
  * Copyright 1998 Massachusetts Institute of Technology
  * Copyright 2012 ADARA Networks, Inc.
+ * Copyright 2017 Dell EMC Isilon
  *
  * Portions of this software were developed by Robert N. M. Watson under
  * contract to ADARA Networks, Inc.
@@ -62,6 +63,7 @@ __FBSDID("$FreeBSD$");
 #include 
 #include 
 #include 
+#include 
 
 #include 
 #include 
@@ -100,6 +102,53 @@ struct ifvlantrunk {
int refcnt;
 };
 
+/*
+ * This macro provides a facility to iterate over every vlan on a trunk with
+ * the assumption that none will be added/removed during iteration.
+ */
+#ifdef VLAN_ARRAY
+#define VLAN_FOREACH(_ifv, _trunk) \
+   size_t _i; \
+   for (_i = 0; _i < VLAN_ARRAY_SIZE; _i++) \
+   if (((_ifv) = (_trunk)->vlans[_i]) != NULL)
+#else /* VLAN_ARRAY */
+#define VLAN_FOREACH(_ifv, _trunk) \
+   struct ifvlan *_next; \
+   size_t _i; \
+   for (_i = 0; _i < (1 << (_trunk)->hwidth); _i++) \
+   LIST_FOREACH_SAFE((_ifv), &(_trunk)->hash[_i], ifv_list, _next)
+#endif /* VLAN_ARRAY */
+
+/*
+ * This macro provides a facility to iterate over every vlan on a trunk while
+ * also modifying the number of vlans on the trunk. The iteration continues
+ * until some condition is met or there are no more vlans on the trunk.
+ */
+#ifdef VLAN_ARRAY
+/* The VLAN_ARRAY case is simple -- just a for loop using the condition. */
+#define VLAN_FOREACH_UNTIL_SAFE(_ifv, _trunk, _cond) \
+   size_t _i; \
+   for (_i = 0; !(_cond) && _i < VLAN_ARRAY_SIZE; _i++) \
+   if (((_ifv) = (_trunk)->vlans[_i]))
+#else /* VLAN_ARRAY */
+/*
+ * The hash table case is more complicated. We allow for the hash table to be
+ * modified (i.e. vlans removed) while we are iterating over it. To allow for
+ * this we must restart the iteration every time we "touch" something during
+ * the iteration, since removal will resize the hash table and invalidate our
+ * current position. If acting on the touched element causes the trunk to be
+ * emptied, then iteration also stops.
+ */
+#define VLAN_FOREACH_UNTIL_SAFE(_ifv, _trunk, _cond) \
+   size_t _i; \
+   bool _touch = false; \
+   for (_i = 0; \
+   !(_cond) && _i < (1 << (_trunk)->hwidth); \
+   _i = (_touch && ((_trunk) != NULL) ? 0 : _i + 1), _touch = false) \
+   if (((_ifv) = LIST_FIRST(&(_trunk)->hash[_i])) != NULL && \
+   (_touch = true))
+#endif /* VLAN_ARRAY */
+
 struct vlan_mc_entry {
struct sockaddr_dl  mc_addr;
SLIST_ENTRY(vlan_mc_entry)  mc_entries;
@@ -122,6 +171,7 @@ struct  ifvlan {
uint16_t ifvm_vid;  /* VLAN ID */
uint8_t ifvm_pcp;   /* Priority Code Point (PCP). */
}   ifv_mib;
+   struct task lladdr_task;
SLIST_HEAD(, vlan_mc_entry) vlan_mc_listhead;
 #ifndef VLAN_ARRAY
LIST_ENTRY(ifvlan) ifv_list;
@@ -172,33 +222,92 @@ static eventhandler_tag ifdetach_tag;
 static eventhandler_tag iflladdr_tag;
 
 /*
- * We have a global mutex, that is used to serialize configuration
- * changes and isn't used in normal packet delivery.
+ * if_vlan uses two module-level locks to allow concurrent modification of vlan
+ * interfaces and (mostly) allow for vlans to be destroyed while they are being
+ * used for tx/rx. To accomplish this in a way that has acceptable performance
+ * and cooperation with other parts of the network stack there is a
+ * non-sleepable rmlock(9) and an sx(9). Both locks are exclusively acquired
+ * when destroying a vlan interface, i.e. when the if_vlantrunk field of struct
+ * ifnet is de-allocated and NULL'd. Thus a reader holding either lock has a
+ * guarantee that the struct ifvlantrunk references a valid vlan trunk.
  *
- * We also have a

Re: svn commit: r322893 - head/bin/dd

2017-08-25 Thread Matt Joras 
On 08/25/2017 10:17, Conrad Meyer wrote:
> This change seems to break buildworld on MIPS:
>
> /home/cem/head.svn/bin/dd/args.c: In function 'f_bs':
> /home/cem/head.svn/bin/dd/args.c:188: warning: format '%zd' expects
> type 'signed size_t', but argument 3 has type 'long int'
> /home/cem/head.svn/bin/dd/args.c: In function 'f_cbs':
> /home/cem/head.svn/bin/dd/args.c:199: warning: format '%zd' expects
> type 'signed size_t', but argument 3 has type 'long int'
> /home/cem/head.svn/bin/dd/args.c: In function 'f_ibs':
> /home/cem/head.svn/bin/dd/args.c:245: warning: format '%zd' expects
> type 'signed size_t', but argument 3 has type 'long int'
> /home/cem/head.svn/bin/dd/args.c: In function 'f_obs':
> /home/cem/head.svn/bin/dd/args.c:266: warning: format '%zd' expects
> type 'signed size_t', but argument 3 has type 'long int'
>
> (Yes, it's odd that the SSIZE_MAX constant has 'long' type.)
>
SSIZE_MAX should have type long, since ssize_t is a long on mips (and
other arches besides i386 and arm).

Re: the build failure, that's in the GCC C format string checking, so
perhaps it's more accurate to say this breaks the (in-tree) GCC build.
%zd is the right format specifier for ssize_t. I guess GCC's format
string checking is getting confused because SSIZE_MAX is a constant that
expands to type long. Perhaps casting to ssize_t would GCC happier, but
that looks rather wrong.

Matt
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r322548 - head/sys/net

2017-08-15 Thread Matt Joras 
Author: mjoras
Date: Tue Aug 15 17:52:37 2017
New Revision: 322548
URL: https://svnweb.freebsd.org/changeset/base/322548

Log:
  Rework vlan(4) locking.
  
  Previously the locking of vlan(4) interfaces was not very comprehensive.
  Particularly there was very little protection against the destruction of
  active vlan(4) interfaces or concurrent modification of a vlan(4)
  interface. The former readily produced several different panics.
  
  The changes can be summarized as using two global vlan locks (an
  rmlock(9) and an sx(9)) to protect accesses to the if_vlantrunk field of
  struct ifnet, in addition to other places where global exclusive access
  is required. vlan(4) should now be much more resilient to the destruction
  of active interfaces and concurrent calls into the configuration path.
  
  PR:   220980
  Reviewed by:  ae, markj, mav, rstone
  Approved by:  rstone (mentor)
  MFC after:4 weeks
  Sponsored by: Dell EMC Isilon
  Differential Revision:https://reviews.freebsd.org/D11370

Modified:
  head/sys/net/if_vlan.c

Modified: head/sys/net/if_vlan.c
==
--- head/sys/net/if_vlan.c  Tue Aug 15 16:39:49 2017(r322547)
+++ head/sys/net/if_vlan.c  Tue Aug 15 17:52:37 2017(r322548)
@@ -1,6 +1,7 @@
 /*-
  * Copyright 1998 Massachusetts Institute of Technology
  * Copyright 2012 ADARA Networks, Inc.
+ * Copyright 2017 Dell EMC Isilon
  *
  * Portions of this software were developed by Robert N. M. Watson under
  * contract to ADARA Networks, Inc.
@@ -63,6 +64,7 @@ __FBSDID("$FreeBSD$");
 #include 
 #include 
 #include 
+#include 
 
 #include 
 #include 
@@ -101,6 +103,53 @@ struct ifvlantrunk {
int refcnt;
 };
 
+/*
+ * This macro provides a facility to iterate over every vlan on a trunk with
+ * the assumption that none will be added/removed during iteration.
+ */
+#ifdef VLAN_ARRAY
+#define VLAN_FOREACH(_ifv, _trunk) \
+   size_t _i; \
+   for (_i = 0; _i < VLAN_ARRAY_SIZE; _i++) \
+   if (((_ifv) = (_trunk)->vlans[_i]) != NULL)
+#else /* VLAN_ARRAY */
+#define VLAN_FOREACH(_ifv, _trunk) \
+   struct ifvlan *_next; \
+   size_t _i; \
+   for (_i = 0; _i < (1 << (_trunk)->hwidth); _i++) \
+   LIST_FOREACH_SAFE((_ifv), &(_trunk)->hash[_i], ifv_list, _next)
+#endif /* VLAN_ARRAY */
+
+/*
+ * This macro provides a facility to iterate over every vlan on a trunk while
+ * also modifying the number of vlans on the trunk. The iteration continues
+ * until some condition is met or there are no more vlans on the trunk.
+ */
+#ifdef VLAN_ARRAY
+/* The VLAN_ARRAY case is simple -- just a for loop using the condition. */
+#define VLAN_FOREACH_UNTIL_SAFE(_ifv, _trunk, _cond) \
+   size_t _i; \
+   for (_i = 0; !(_cond) && _i < VLAN_ARRAY_SIZE; _i++) \
+   if (((_ifv) = (_trunk)->vlans[_i]))
+#else /* VLAN_ARRAY */
+/*
+ * The hash table case is more complicated. We allow for the hash table to be
+ * modified (i.e. vlans removed) while we are iterating over it. To allow for
+ * this we must restart the iteration every time we "touch" something during
+ * the iteration, since removal will resize the hash table and invalidate our
+ * current position. If acting on the touched element causes the trunk to be
+ * emptied, then iteration also stops.
+ */
+#define VLAN_FOREACH_UNTIL_SAFE(_ifv, _trunk, _cond) \
+   size_t _i; \
+   bool _touch = false; \
+   for (_i = 0; \
+   !(_cond) && _i < (1 << (_trunk)->hwidth); \
+   _i = (_touch && ((_trunk) != NULL) ? 0 : _i + 1), _touch = false) \
+   if (((_ifv) = LIST_FIRST(&(_trunk)->hash[_i])) != NULL && \
+   (_touch = true))
+#endif /* VLAN_ARRAY */
+
 struct vlan_mc_entry {
struct sockaddr_dl  mc_addr;
SLIST_ENTRY(vlan_mc_entry)  mc_entries;
@@ -123,6 +172,7 @@ struct  ifvlan {
uint16_t ifvm_vid;  /* VLAN ID */
uint8_t ifvm_pcp;   /* Priority Code Point (PCP). */
}   ifv_mib;
+   struct task lladdr_task;
SLIST_HEAD(, vlan_mc_entry) vlan_mc_listhead;
 #ifndef VLAN_ARRAY
LIST_ENTRY(ifvlan) ifv_list;
@@ -173,33 +223,92 @@ static eventhandler_tag ifdetach_tag;
 static eventhandler_tag iflladdr_tag;
 
 /*
- * We have a global mutex, that is used to serialize configuration
- * changes and isn't used in normal packet delivery.
+ * if_vlan uses two module-level locks to allow concurrent modification of vlan
+ * interfaces and (mostly) allow for vlans to be destroyed while they are being
+ * used for tx/rx. To accomplish this in a way that has acceptable performance
+ * and cooperation with other parts of the network stack there is a
+ * non-sleepable rmlock(9) and an sx(9). Both locks are exclusively acquired
+ * when destroying a vlan interface, i.e. when the if_vlantrunk field of struct
+ * ifnet is de-allocated and NULL'd.

svn commit: r322062 - head/sbin/ifconfig

2017-08-04 Thread Matt Joras 
Author: mjoras
Date: Fri Aug  4 21:06:47 2017
New Revision: 322062
URL: https://svnweb.freebsd.org/changeset/base/322062

Log:
  Selectively print "hwaddr" from ifconfig(8).
  
  ifconfig(8) printing the hwaddr is only really useful if it differs from
  the link layer address.
  
  Reported by:  jhb
  Reviewed by:  rpokala
  Approved by:  rstone (mentor)
  Differential Revision:https://reviews.freebsd.org/D11777

Modified:
  head/sbin/ifconfig/af_link.c

Modified: head/sbin/ifconfig/af_link.c
==
--- head/sbin/ifconfig/af_link.cFri Aug  4 20:24:23 2017
(r322061)
+++ head/sbin/ifconfig/af_link.cFri Aug  4 21:06:47 2017
(r322062)
@@ -108,7 +108,15 @@ link_status(int s __unused, const struct ifaddrs *ifa)
if (rc != 0) {
return;
}
-   if (memcmp(ifr.ifr_addr.sa_data, laggaddr, 
sdl->sdl_alen) == 0) {
+
+   /*
+* If this is definitely a lagg device or the hwaddr
+* matches the link addr, don't bother.
+*/
+   if (memcmp(ifr.ifr_addr.sa_data, laggaddr,
+   sdl->sdl_alen) == 0 ||
+   memcmp(ifr.ifr_addr.sa_data, LLADDR(sdl),
+   sdl->sdl_alen) == 0) {
return;
}
ether_format = ether_ntoa((const struct ether_addr *)
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r321804 - head/usr.bin/calendar/calendars

2017-07-31 Thread Matt Joras 
Author: mjoras
Date: Mon Jul 31 18:10:10 2017
New Revision: 321804
URL: https://svnweb.freebsd.org/changeset/base/321804

Log:
  Add myself to the calendar.
  
  Reported by:  mckusick
  Approved by:  rstone (mentor)
  Differential Revision:https://reviews.freebsd.org/D11797

Modified:
  head/usr.bin/calendar/calendars/calendar.freebsd

Modified: head/usr.bin/calendar/calendars/calendar.freebsd
==
--- head/usr.bin/calendar/calendars/calendar.freebsdMon Jul 31 17:57:54 
2017(r321803)
+++ head/usr.bin/calendar/calendars/calendar.freebsdMon Jul 31 18:10:10 
2017(r321804)
@@ -374,6 +374,7 @@
 11/15  Lars Engels <l...@freebsd.org> born in Hilden, Nordrhein-Westfalen, 
Germany, 1980
 11/15  Tijl Coosemans <t...@freebsd.org> born in Duffel, Belgium, 1983
 11/16  Jose Maria Alcaide Salinas <j...@freebsd.org> born in Madrid, Spain, 
1962
+11/16  Matt Joras <mjo...@freebsd.org> born in Evanston, Illinois, United 
States, 1992
 11/17  Ralf S. Engelschall <r...@freebsd.org> born in Dachau, Bavaria, 
Germany, 1972
 11/18  Thomas Quinot <tho...@freebsd.org> born in Paris, France, 1977
 11/19  Konstantin Belousov <k...@freebsd.org> born in Kiev, USSR, 1972
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r321301 - head/share/misc

2017-07-20 Thread Matt Joras 
Author: mjoras
Date: Thu Jul 20 18:14:27 2017
New Revision: 321301
URL: https://svnweb.freebsd.org/changeset/base/321301

Log:
  Add myself and mentor line to committers-src.dot.
  
  Approved by:  rstone (mentor)
  Differential Revision:https://reviews.freebsd.org/D11672

Modified:
  head/share/misc/committers-src.dot

Modified: head/share/misc/committers-src.dot
==
--- head/share/misc/committers-src.dot  Thu Jul 20 18:02:25 2017
(r321300)
+++ head/share/misc/committers-src.dot  Thu Jul 20 18:14:27 2017
(r321301)
@@ -244,6 +244,7 @@ melifaro [label="Alexander V. Chernikov\nmelifaro@Free
 mizhka [label="Michael Zhilin\nmiz...@freebsd.org\n2016/07/19"]
 mjacob [label="Matt Jacob\nmja...@freebsd.org\n1997/08/13"]
 mjg [label="Mateusz Guzik\n...@freebsd.org\n2012/06/04"]
+mjoras [label="Matt Joras\nmjo...@freebsd.org\n2017/07/12"]
 mlaier [label="Max Laier\nmla...@freebsd.org\n2004/02/10"]
 mmel [label="Michal Meloun\nm...@freebsd.org\n2015/11/01"]
 monthadar [label="Monthadar Al Jaberi\nmontha...@freebsd.org\n2012/04/02"]
@@ -720,6 +721,7 @@ rrs -> jchandra
 rrs -> tuexen
 
 rstone -> markj
+rstone -> mjoras
 
 ru -> ceri
 ru -> cjc
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"