On Tue, Jan 30, 2007 at 12:24:58PM -0800, Sunay Tripathi wrote:
> > Beyond that, will inbound traffic scatter across multiple CPUs? Consider a
> > node that has multiple inbound IPsec SAs. Each inbound SA will have one or
> > more individual flows inside it, AND if all of them hammer one CPU their
> > high-overhead decrypt/verify operations cannot be shared among multiple
> > CPUs.
>
> Yes, you can scatter this across multiple CPUs. Again using dladm/flowadm,
What you refer to as "flowadm" is called "netrcm" on your Dec. 2006 man
pages. Is that what you mean?
> you will be able to specify a '-C' option for CPU list which can apply to
> both VNIC or a flow. The way ti works is that after classification, the
> upcall function is a fanout function which takes as an argument a set of
> soft rings with their own worker thread boudn to CPUs specified via '-C'
> option.
Yeah. That part tracks.
> I don't think we have code to do optimal fanout based on SA but that should
> be easy to do.
And "based on SA" is why I brought up this thread.
In the absence of other data (which is actually VERY hard to get without
getting key management - e.g. IKE - involved) all you really want to do is:
* Assume inbound SPI generation is random. This is almost always the
case apart from manually keyed IPsec SAs.
* Send an inbound IPsec packet to a specific processor based on the
SPI value modulo the number of available CPUs. Assuming a nice
power-of-2, it's a masking operation.
Is that simple enough to shoehorn into the Crossbow way of doing things? If
you already assign CPUs based on TCP/UDP ports, it should be trivial, as the
IPsec SPI is at the same offset in the packet as TCP/UDP ports.
> > Does crossbow improve this situation? Or are we already there w.r.t.
> > sharing in bound load?
To answer my own question based on what you said - crossbow might improve
this situation if it can fanout based on SA per my notes above.
> More interesting would be setting up a flow/VNIC and a CPU list to deal
> with encrypted traffic but feed it back to classifier after decryption so
> that policies based on IP address, ports etc can still be honoured and
> decryption can be done on separate set of CPUs altogether. With N2 crypto
> offload engines, it would be a interesting comboniation.
If one's IPsec policy (and subsequent SAs) have port/protocol selectors
involved, one could use that information to look into the existing classifier
and setup a post-IPsec-processing path that honors the classifier's entries
that is cached with the SA. (This is a variant on what we do today with
caching *outbound* SAs with a conn_t.)
Thanks for the answer, and if you can get IPsec multi-CPU fanout going... heh
heh... combine that with N2 like you say, and inbound IPsec processing
doesn't become the single-CPU hog it once was.
Dan
