On Tue, Oct 12, 2010 at 10:54 AM, Tzachi Dar <[email protected]> wrote: > There is no interrupt coalescing on winof although the hw does support that > (I assume that you are running connectx). > > You can verify if the number of interrupts is big by running perfmon. Select > the cpu, and choose number of interrupts per second. > > Look at this when your test is running and when it is not.
Although there are some Heisenberg issues w/ this test (perfmon locks up trying to measure cores that are pegged), I was able to confirm that all the interrupts are hitting these two cores, and assume each core is getting the interrupts for one HCA. Thaks, Chris > > Thanks > Tzachi > >> -----Original Message----- >> From: Chris Worley [mailto:[email protected]] >> Sent: Tuesday, October 12, 2010 6:49 PM >> To: Tzachi Dar >> Cc: [email protected] >> Subject: Re: [ofw] Any interrupt coalescing in WinOF? >> >> On Tue, Oct 12, 2010 at 10:34 AM, Tzachi Dar <[email protected]> >> wrote: >> > What test/ulp are you using? >> >> IOMeter with one worker atop each of 4 SRP-imported/multipathed >> (round-robin) drives and a queue depth of 64. I'm seeing ~450K random >> 4KB write IOPS. >> >> With two HCA's, both running the latest firmware (as of Friday), I see >> two cores on one NUMA node totally pegged/consumed, and affinity is >> set on the Dynamo's to avoid these cores... so something is either >> polling or interrupt handling on these. >> >> I'm running the latest 2.3RC4 WinOF. >> >> Thanks, >> >> Chris >> > >> > Thanks >> > Tzachi >> > >> >> -----Original Message----- >> >> From: [email protected] [mailto:ofw- >> >> [email protected]] On Behalf Of Chris Worley >> >> Sent: Tuesday, October 12, 2010 6:33 PM >> >> To: [email protected] >> >> Subject: [ofw] Any interrupt coalescing in WinOF? >> >> >> >> I'm seeing the interrupt handlers completely consuming a CPU and >> >> becoming a bottleneck, and no interrupt load balancing across CPU's >> >> (except w/ multiple HCA's, each HCA consumes one CPU w/ interrupts). >> >> >> >> Is interrupt coalescing manually tunable in WinOF? >> >> >> >> Or, am I totally off-base, and seeing polling and not interrupt >> >> behavior? >> >> >> >> Thanks, >> >> >> >> Chris >> >> _______________________________________________ >> >> ofw mailing list >> >> [email protected] >> >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw >> > > _______________________________________________ ofw mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
