Re: DPDK and energy efficiency

2021-03-05 Thread Eric Kuhnke
> Server PS maximum input wattage is 900W. Present draw of 2.0A @ 208V is ~420W, so 420/900 = 46.67% But in the real world an R640 would *never* draw 900W. Even if you were to load it up with the maximal CPU configuration (2 x 125W TDP CPU per socket), a full load of 2.5" 15K spinning drives,

Re: DPDK and energy efficiency

2021-03-05 Thread Brian Knight via NANOG
On 2021-03-05 15:40, Eric Kuhnke wrote: For comparison purposes, I'm curious about the difference in wattage results between: a) Your R640 at 420W running DPDK b) The same R640 hardware temporarily booted from a Ubuntu server live USB, in which some common CPU stress and memory disk/IO

Re: DPDK and energy efficiency

2021-03-05 Thread Eric Kuhnke
For comparison purposes, I'm curious about the difference in wattage results between: a) Your R640 at 420W running DPDK b) The same R640 hardware temporarily booted from a Ubuntu server live USB, in which some common CPU stress and memory disk/IO benchmarks are being run to intentionally load

Re: DPDK and energy efficiency

2021-03-05 Thread Eric Kuhnke
That was an unfortunate typo on my part, I meant to write "isn't excessively difficult..." Some real world examples of specific models of CPU + motherboard + PCI-E NIC combinations with wattage figures at idle load, average load and maximal load would be useful for comparison purposes. On Fri,

Re: DPDK and energy efficiency

2021-03-05 Thread Brian Knight via NANOG
On 2021-03-05 12:22, Etienne-Victor Depasquale wrote: Sure, here goes: https://www.surveymonkey.com/results/SM-BJ9FCT6K9/ Thanks for sharing these results. We run DPDK workloads (Cisco nee Viptela vEdge Cloud) on ESXI. Fwiw, a quick survey of a few of our Dell R640s running mostly vEdge

Re: DPDK and energy efficiency

2021-03-05 Thread Etienne-Victor Depasquale
Sure, here goes: https://www.surveymonkey.com/results/SM-BJ9FCT6K9/ Cheers, Etienne On Fri, Mar 5, 2021 at 5:06 PM Tom Hill wrote: > On 04/03/2021 18:20, Etienne-Victor Depasquale wrote: > > *SECTION 2: Survey results* > > I don't see the embedded images, and there's no way to show them

Re: DPDK and energy efficiency

2021-03-05 Thread Tom Hill
On 05/03/2021 00:26, Eric Kuhnke wrote: > A great deal of this discussion could be resolved by the use of a $20 > in-line 120VAC watt meter [1] plugged into something as simple as a $500 > 1U server with some of the DPDK-enabled network cards connected to its > PCI-E bus, running DANOS. I'm

Re: DPDK and energy efficiency

2021-03-05 Thread Tom Hill
On 04/03/2021 18:20, Etienne-Victor Depasquale wrote: > *SECTION 2: Survey results* I don't see the embedded images, and there's no way to show them inline. For the sake of simplicity/sharing, are these results presented anywhere on a web page? :) Regards, -- Tom

Re: DPDK and energy efficiency

2021-03-04 Thread Eric Kuhnke
A great deal of this discussion could be resolved by the use of a $20 in-line 120VAC watt meter [1] plugged into something as simple as a $500 1U server with some of the DPDK-enabled network cards connected to its PCI-E bus, running DANOS. Characterizing the idle load, average usage load, and

Re: DPDK and energy efficiency

2021-03-04 Thread Etienne-Victor Depasquale
*TL;DR - DPDK applications embody the phrase caveat emptor.* As Robert Bays put it: "Please ask your open source dev and/or vendor of choice to verify." On the other hand, I do not recommend taking the following (citing Robert Bays again) for granted: "But the reality is [open source projects

Re: DPDK and energy efficiency

2021-02-27 Thread Etienne-Victor Depasquale
Just a quick note to say that I've closed the survey. I haven't published the results yet as I said that I would write notes necessary as a preamble to correctly inform potential readers, and these notes are taking longer to write than I have time available. Cheers, Etienne On Wed, Feb 24,

Re: DPDK and energy efficiency

2021-02-24 Thread Etienne-Victor Depasquale
I think I need to calm this thread down. I'm a researcher, and my interest is in the truth, not in my opinion. I've read some facts in this thread that are necessary as a prerequisite to the publication of the results on Friday. I do want to ensure that no future reader is misinformed and will

Re: DPDK and energy efficiency

2021-02-24 Thread Robert Bays
To the nanog community, I’m sorry to have dragged this conversation out further. I'm only responding to this because there are a significant number of open source projects and commercial products that use DPDK, or similar userspace network environment in their implementations. The statements

Re: DPDK and energy efficiency

2021-02-24 Thread Douglas Fischer
The statement used on the survey "Are you aware that use of DPDK on a processor core keeps utilization at 100% regardless of packet activity?" can be easily distorted and badly used. I sincerely do not agree with the approach of presuming and declaring "DPDK spent too much power". Mainly because

Re: DPDK and energy efficiency

2021-02-23 Thread Etienne-Victor Depasquale
Hello Robert, Your statement that DPDK “keeps utilization at 100% regardless of packet > activity” is just not correct. You further pre-suppose "widespread DPDK's > core operating inefficiency” without any data to backup the operating > inefficacy assertion. > This statement is incorrect. I

Re: DPDK and energy efficiency

2021-02-23 Thread Robert Bays
Hi Etienne, Your statement that DPDK “keeps utilization at 100% regardless of packet activity” is just not correct. You further pre-suppose "widespread DPDK's core operating inefficiency” without any data to backup the operating inefficacy assertion. Your statements, taken at face value,

Re: DPDK and energy efficiency

2021-02-23 Thread Etienne-Victor Depasquale
> > This is way too deep in the weeds of developing with the DPDK > libraries for your audience here to have much in the way of useful > comment. This is an operators group. > Fair enough, and thank you for stepping on the brakes :) Honestly, I didn't intend to get embroiled in this. The

Re: DPDK and energy efficiency

2021-02-23 Thread Etienne-Victor Depasquale
> > This comes from OVS code and shows OVS thread spinning, not DPDK PMD. > Blame the OVS application for not using e.g. _mm_pause() and burning > the CPU like crazy. > OK, I'm citing a bit more from the same reference: *"By tracing back to the function’s caller * *in the PMD thread main(void

Re: DPDK and energy efficiency

2021-02-23 Thread William Herrin
On Tue, Feb 23, 2021 at 2:22 PM Etienne-Victor Depasquale wrote: >> DPDK doesn't inherently do much in the way of power management. > > I agree - it doesn't. That's not what it was made for. > >> Note that DPDK applications are usually intended to run in very-high > > data rate environments

Re: DPDK and energy efficiency

2021-02-23 Thread Etienne-Victor Depasquale
Oh dear ... instead of "and in [6]", I should have written "and in [3]". On Tue, Feb 23, 2021 at 11:21 PM Etienne-Victor Depasquale wrote: > DPDK doesn't inherently do much in the way of power management. >> > I agree - it doesn't. That's not what it was made for. > > Note that DPDK

Re: DPDK and energy efficiency

2021-02-23 Thread Etienne-Victor Depasquale
> > DPDK doesn't inherently do much in the way of power management. > I agree - it doesn't. That's not what it was made for. Note that DPDK applications are usually intended to run in very-high data rate environments where no gains are likely to be realized by avoiding a busy-wait loop. That's

Re: DPDK and energy efficiency

2021-02-23 Thread William Herrin
On Mon, Feb 22, 2021 at 11:24 PM Etienne-Victor Depasquale wrote: >> >> Beyond RX/TX CPU affinity, in DANOS you can further tune power consumption >> by changing the adaptive polling rate. It doesn’t, per the survey, "keep >> utilization at 100% regardless of packet activity.” > > Robert, you

Re: DPDK and energy efficiency

2021-02-23 Thread Pawel Malachowski
> > No, it is not PMD that runs the processor in a polling loop. > > It is the application itself, thay may or may not busy loop, > > depending on application programmers choice. > > From one of my earlier references [2]: > > "we found that a poll mode driver (PMD) > thread accounted for

Re: DPDK and energy efficiency

2021-02-23 Thread Etienne-Victor Depasquale
> > Probably yeah. Have you assessed the lifetime cost of running a > multicore CPU at 100% vs at 10%, particularly as you're likely to have > multiples of these devices in operation? > Spot on. On Tue, Feb 23, 2021 at 6:07 PM Nick Hilliard wrote: > Shane Ronan wrote on 23/02/2021 16:59: > >

Re: DPDK and energy efficiency

2021-02-23 Thread Nick Hilliard
Shane Ronan wrote on 23/02/2021 16:59: For use cases where DPDK matters, are you really concerned with power consumption? Probably yeah. Have you assessed the lifetime cost of running a multicore CPU at 100% vs at 10%, particularly as you're likely to have multiples of these devices in

Re: DPDK and energy efficiency

2021-02-23 Thread Shane Ronan
For use cases where DPDK matters, are you really concerned with power consumption? On Tue, Feb 23, 2021 at 11:48 AM Nick Hilliard wrote: > Etienne-Victor Depasquale wrote on 23/02/2021 16:03: > > "we found that a poll mode driver (PMD) > > thread accounted for approximately 99.7 percent > >

Re: DPDK and energy efficiency

2021-02-23 Thread Nick Hilliard
Etienne-Victor Depasquale wrote on 23/02/2021 16:03: "we found that a poll mode driver (PMD) thread accounted for approximately 99.7 percent CPU occupancy (a full core utilization)." interrupt-driven network drivers generally can't compete with polled mode drivers at higher throughputs on

Re: DPDK and energy efficiency

2021-02-23 Thread Etienne-Victor Depasquale
> > No, it is not PMD that runs the processor in a polling loop. > It is the application itself, thay may or may not busy loop, > depending on application programmers choice. > >From one of my earlier references [2]: "we found that a poll mode driver (PMD) thread accounted for approximately 99.7

Re: DPDK and energy efficiency

2021-02-23 Thread Pawel Malachowski
Dnia Mon, Feb 22, 2021 at 12:45:52PM +0100, Etienne-Victor Depasquale napisał(a): > Every research paper I've read indicates that, regardless of whether it has > packets to process or not, DPDK PMDs (poll-mode drivers) prevent the CPU > from falling into an LPI (low-power idle). > > When it has

Re: DPDK and energy efficiency

2021-02-22 Thread Etienne-Victor Depasquale
Sorry, last line should have been: "intended to get an impression of how widespread ***knowledge of*** DPDK's core operating inefficiency is", not: "intended to get an impression of how widespread DPDK's core operating inefficiency is" On Tue, Feb 23, 2021 at 8:22 AM Etienne-Victor Depasquale

Re: DPDK and energy efficiency

2021-02-22 Thread Etienne-Victor Depasquale
> > Beyond RX/TX CPU affinity, in DANOS you can further tune power consumption > by changing the adaptive polling rate. It doesn’t, per the survey, "keep > utilization at 100% regardless of packet activity.” > Robert, you seem to be conflating DPDK with DANOS' power control algorithms that

Re: DPDK and energy efficiency

2021-02-22 Thread Jared Geiger
> “hard” partitioning of cores is a key part of the DSA (domain-specific > architecture) here. > > > > Sent from my Windows 10 device > > > > *From: *Jared Geiger > *Sent: *Monday, 22 February 2021 20:53 > *To: *NANOG > *Subject: *Re: DPDK and energy effi

RE: DPDK and energy efficiency

2021-02-22 Thread Etienne Depasquale
of the DSA (domain-specific architecture) here. Sent from my Windows 10 device From: Jared Geiger Sent: Monday, 22 February 2021 20:53 To: NANOG Subject: Re: DPDK and energy efficiency DANOS lets you specify how many dataplane cores you use versus control plane cores. So if you put a 16 core host

Re: DPDK and energy efficiency

2021-02-22 Thread Robert Bays
Beyond RX/TX CPU affinity, in DANOS you can further tune power consumption by changing the adaptive polling rate. It doesn’t, per the survey, "keep utilization at 100% regardless of packet activity.” Adaptive polling changes in DPDK optimize for tradeoffs between power consumption,

Re: DPDK and energy efficiency

2021-02-22 Thread Jared Geiger
DANOS lets you specify how many dataplane cores you use versus control plane cores. So if you put a 16 core host in to handle 2GB of traffic, you can adjust the dataplane worker cores as needed. Control plane cores don't stay at 100% utilization. I use that technique plus DANOS runs on VMware

Re: DPDK and energy efficiency

2021-02-22 Thread Etienne-Victor Depasquale
I forgot to point out that on Friday 26th, I'll share the results collected through a link or a series of screenshots. Cheers, Etienne On Mon, Feb 22, 2021 at 2:15 PM Pawel Malachowski < pawmal-na...@freebsd.lublin.pl> wrote: > Dnia Mon, Feb 22, 2021 at 01:01:45PM +0100, Etienne-Victor

Re: DPDK and energy efficiency

2021-02-22 Thread Pawel Malachowski
Dnia Mon, Feb 22, 2021 at 01:01:45PM +0100, Etienne-Victor Depasquale napisał(a): > It is, after all, Intel's response to the problem of general-purpose > scheduling of its processors - which prevents the processor from being > viable under high networking loads. It totally makes sense to busy

Re: DPDK and energy efficiency

2021-02-22 Thread Etienne-Victor Depasquale
> > It consumes 100% only if you busy poll (which is the default approach). > Precisely. It is, after all, Intel's response to the problem of general-purpose scheduling of its processors - which prevents the processor from being viable under high networking loads. Cheers, Etienne On Mon, Feb

Re: DPDK and energy efficiency

2021-02-22 Thread Etienne-Victor Depasquale
Here are a few references. Strictly speaking, DPDK and SR-IOV are orthogonal. DPDK is intended to facilitate cloud-native operation through hardware independence. SR-IOV presumes SR-IOV-compliant hardware. [1] Z. Xu, F. Liu, T. Wang, and H. Xu, “Demystifying the energy efficiency of Network

Re: DPDK and energy efficiency

2021-02-22 Thread Pawel Malachowski
Dnia Mon, Feb 22, 2021 at 08:33:35AM -0300, Douglas Fischer napisał(a): > But IMHO, the questions do not cover the actual reality of DPDK. > That característic of "100% CPU" depends on several aspects, like: > - How old are the hardware on DPDK. > - What type of DPDK Instructions are made(Very

Re: DPDK and energy efficiency

2021-02-22 Thread Etienne-Victor Depasquale
> > The way I saw, the questions induce the public to conclude that DPDK > ALWAYS has 100% CPU usage, which is not true. I don't concur. Every research paper I've read indicates that, regardless of whether it has packets to process or not, DPDK PMDs (poll-mode drivers) prevent the CPU from

Re: DPDK and energy efficiency

2021-02-22 Thread Douglas Fischer
I'm very happy to see interest in DPDK and power consumption. But IMHO, the questions do not cover the actual reality of DPDK. That característic of "100% CPU" depends on several aspects, like: - How old are the hardware on DPDK. - What type of DPDK Instructions are made(Very Dynamic as

DPDK and energy efficiency

2021-02-21 Thread Etienne-Victor Depasquale
Hello folks, I've just followed a thread regarding use of CGNAT and noted a suggestion (regarding DANOS) that includes use of DPDK. As I'm interested in the breadth of adoption of DPDK, and as I'm a researcher into energy and power efficiency, I'd love to hear your feedback on your use of power