--- On Thu, 6/20/13, Andre Oppermann wrote: > From: Andre Oppermann > Subject: Re: hw.igb.num_queues default > To: "Eugene Grosbein" > Cc: "freebsd-net@freebsd.org" , "Eggert, Lars" , "Jack Vogel" > Date: Thursday, June 20, 2013, 10:29 AM > On 20.06.2013 15:37, Eugene Grosbein > wrote: > > On 20.06.2013 17:34, Eggert, Lars wrote: > > > >> real memory = 8589934592 (8192 MB) > >> avail memory = 8239513600 (7857 MB) > > > >> By default, the igb driver seems to set up one > queue per detected CPU. Googling around, people seemed to > suggest that limiting the number of queues makes things work > better. I can confirm that setting hw.igb.num_queues=2 seems > to have fixed the issue. (Two was the first value I tried, > maybe other values other than 0 would work, too.) > >> > >> In order to uphold POLA, should the igb driver > maybe default to a conservative value for hw.igb.num_queues > that may not deliver optimal performance, but at least works > out of the box? > > > > Or, better, make nmbclusters auto-tuning smarter, if > any. > > I mean, use more nmbclusters for machines with large > amounts of memory. > > That has already been done in HEAD. > > The other problem is the pre-filling of the large rings for > all queues > stranding large amounts of mbuf clusters. OpenBSD > starts with a small > number of filled mbufs in the RX ring and then dynamically > adjusts the > number upwards if there is enough traffic to maintain deep > buffers. I > don't know if it always quickly scales in practice though.
You're probably not running with 512MB these days, so pre-filling isn't much of an issue. 4 queues is only 8MB of ram with 1024 descriptors per queue, and 4MB with 512. Think about the # of queues issue. In order to have acceptable latency, you need to do 6k-10k interrupts per second per queue. So with 4 queues you have to process 40K ints/second and with 2 you only process 20k. For a gig link 2 queues is much more efficient. "Spreading" for the sake of spreading is more about Intel marketing than it is about practical computing. BC BC _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"