On Thu, 23 Jan 2020, Gleb Smirnoff wrote:
On Thu, Jan 23, 2020 at 08:33:58PM -0500, Ryan Stone wrote:
R> > Because at interrupt level we can batch multiple packets in a single epoch.
R> > This speeds up unfiltered packet forwarding performance by 5%.
R> >
R> > With driver level pfil hooks I would claim even more improvement, because
before
R> > the change we needed to enter epoch twice - once for filtering, than for
ether_input.
R> >
R> > Epoch isn't a layer, it is a synchronisation primitive, so I disagree about
R> > statement on layering violation.
R>
R> Epoch is a synchronization primitive, but the net_epoch is absolutely
R> a part of the networking layer. If we need better batching then the
R> correct solution is to introduce a batched interface for drivers to
R> push packets up the stack, not to mess around at the interrupt layer.
Such interface of course would be valuable but will not cover case
when an interrupt arrives during processing of previous one. So its
batching possiblities are limited compared to interrupt level batching.
And I already noted that ether_input() isn't the only way to enter
the network stack.
R> Note that the only reason why this works for mlx4/mlx5 is that
R> linuxkpi *always* requests a INTR_TYPE_NET no matter what driver is
R> running. This means that all drm graphics driver interrupts are now
R> running under the net_epoch:
R>
R>
https://svnweb.freebsd.org/base/head/sys/compat/linuxkpi/common/include/linux/interrupt.h?revision=352205&view=markup#l103
The historical reason is that linuxkpi was originally made to support ofed
and there was no real way to get this information from the driver.
Well, it is not my fault that a video driver requests INTR_TYPE_NET
interrupt. I mean, you can't put this as a rationale against using
network epoch for all interrrupts that declare theirselves as network
interrupts. However, this is harmless.
While we don't have a policy strictly requiring reviews it is the norm to
have substantial changes socialized and reviewed. I appreciate the work
that you are doing but it likely should've been discussed somewhere
more publicly. I apologized if I missed it but I don't see reference to
anything.
Architecturally I am more concerned with the coarseness of net_epoch and
the duration of hold becoming a resource utilization problem in high
turn-over workloads. Like short connection tcp. Has anyone done
substantial testing here? epoch as it is today will hold every free
callback for a minimum of several clock ticks and a maximum of 2x the
duration of the longest epoch section time. With preemption, etc. this
could be 100s of ms of PCBs held.
Thanks,
Jeff
--
Gleb Smirnoff
_______________________________________________
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
