Our test traffic as all UDP, because that is the easiest traffic to
generate at such high levels. We are using this as a transparent
firewall, so we have the two interfaces bridged.
ethtool -S shows that fdir_match is 0, but that is expected since all of
our traffic is routed/bridged.
We currently aren't using vlans, but testing that scenario is on my
to-do list.
Also, I think our problem with only using one tx interrupt is that I
didn't set CPU affinity on the interrupts before looking at the stats.
I've noticed that the newer drivers only have a single interrupt for
RX/TX instead of one for each. Is there any way to get the old
behavior? I think we could still make use of 24 processors by setting
RSS to 12,12 and binding each direction to a different processor (ie.
eth0 rx0 and eth1 tx0 to CPU0, eth0 tx0 and eth1 rx0 to CPU1, etc). We
haven't been able to test this configuration since the in-kernel drivers
don't support the RSS option and the new drivers don't create separate
interrupts.
--
Eldon Koyle
--
"Give me enough medals, and I'll win any war."
-- Napoleon
On Dec 13 16:29-0800, Brandeburg, Jesse wrote:
> The 16 queue limit is a function of the adapter only doing RSS traffic
> direction for receives.
>
> RSS has a hard limit of only 16 queues being able to be picked out from
> the lower bits of the hash. Does the flow director statistics in
> ethtool -S say anything interesting (like all misses)
>
> what type of traffic are you transferring? If it is UDP we will only do
> RSS over the IP headers, and if it is routed traffic we can only do RSS
> typically. Are you using vlans?
>
> if it is TCP (endpoint) traffic being serviced by the server (say by
> apache) then flow director should kick in and parse the packets during
> transmit and make sure the receives for that flow come back to the core
> which transmitted that flow.
>
> Jesse
>
> -----Original Message-----
> From: Eldon Koyle [mailto:[email protected]]
> Sent: Friday, December 10, 2010 3:36 PM
> To: Waskiewicz Jr, Peter P
> Cc: [email protected]
> Subject: Re: [E1000-devel] ixgbe with many processors
>
> We have tried the in-kernel drivers from 2.6.32 and 2.6.36 running on
> Debian unstable. We also tried version 3.0.14 downloaded from
> intel.com. It looks like 3.1.15 was released a few days ago, do you
> think that will make any difference?
>
> It does create 24 RxTx interrupts for each interface, we just aren't
> seeing any traffic come in past the first 16.
>
> --
> Eldon Koyle
> --
> He is the best of men who dislikes power.
> -- Mohammed
>
> On Dec 10 14:48-0800, Waskiewicz Jr, Peter P wrote:
> > It sounds like you're using an older kernel, perhaps something based
> on
> > RHEL5? If this is the case, then you're defaulting to using RSS for
> receive
> > side filtering, which is limited to 16 queues.
> >
> > PJ Waskiewicz
> > [email protected]
> >
> >
> > -----Original Message-----
> > From: Eldon Koyle [mailto:[email protected]]
> > Sent: Friday, December 10, 2010 2:08 PM
> > To: [email protected]
> > Subject: [E1000-devel] ixgbe with many processors
> >
> > We are evaluating some new hardware to be used as firewalls.
> >
> > We are using a dual-port 82599-based card.
> >
> > Currently, we have an AMD box with 2x12-core processors in it, and an
> > intel box with 2x6-core processors and hyperthreading. The ixgbe
> driver
> > creates 24 interrupts, but we only recieve packets on 16 of them. We
> > have tried both the in-kernel and the latest stable drivers from
> intel.
> > The documentation claims that these cards support up to 64(?)
> > interrupts. Am I doing something wrong?
> >
> > (Also, I have been unable to subscribe to this list, so please CC me.)
> >
> > --
> > Eldon Koyle
> > --
> > BOFH excuse #273:
> > The cord jumped over and hit the power switch.
> >
> >
> ------------------------------------------------------------------------
> ----
> > --
> > Oracle to DB2 Conversion Guide: Learn learn about native support for
> PL/SQL,
> > new data types, scalar functions, improved concurrency, built-in
> packages,
> > OCI, SQL*Plus, data movement tools, best practices and more.
> > http://p.sf.net/sfu/oracle-sfdev2dev
> > _______________________________________________
> > E1000-devel mailing list
> > [email protected]
> > https://lists.sourceforge.net/lists/listinfo/e1000-devel
> > To learn more about Intel® Ethernet, visit
> > http://communities.intel.com/community/wired
>
> ------------------------------------------------------------------------
> ------
> Oracle to DB2 Conversion Guide: Learn learn about native support for
> PL/SQL,
> new data types, scalar functions, improved concurrency, built-in
> packages,
> OCI, SQL*Plus, data movement tools, best practices and more.
> http://p.sf.net/sfu/oracle-sfdev2dev
> _______________________________________________
> E1000-devel mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/e1000-devel
> To learn more about Intel® Ethernet, visit
> http://communities.intel.com/community/wired
------------------------------------------------------------------------------
Lotusphere 2011
Register now for Lotusphere 2011 and learn how
to connect the dots, take your collaborative environment
to the next level, and enter the era of Social Business.
http://p.sf.net/sfu/lotusphere-d2d
_______________________________________________
E1000-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit
http://communities.intel.com/community/wired
