Re: Interrupt affinity cannot be set on Mellanox card

Hi Timothy, I mainly consulted this: http://public.dhe.ibm.com/software/dw/linux390/perf/network_tuning.pdf I know network tuning is not easy, there're so many factors. Deep understanding of network protocols and OS layer is needed to do a good job. But this case was not so complicated, it is

Re: Interrupt affinity cannot be set on Mellanox card

James, Have you consulted any specific LinuxONE or Linux on Z network performance tuning documentation yet? There are many other parameters and settings that can influence performance.

Re: Interrupt affinity cannot be set on Mellanox card

I learned a lot from your discussion. Thanks. My question was from a network micro benchmark test which simulated OLTP workloads generating packet less than 200 bytes typically. And our result was only half of the x86's. It's easy to re-produce using uperf/netperf by sending huge amount of small

Re: Interrupt affinity cannot be set on Mellanox card

On Fri, 17 Nov 2017, Alan Altmark wrote: > So what you're saying, Sebastian, is that (a) fine-grained control over > what devices interrupt what CPUs is not in the Z architecture, (b) it's > not in there for a good reason, (c) providing such a capability would > adversely affect overall system

Re: Interrupt affinity cannot be set on Mellanox card

Alan Altmark wrote: >It might, however, be an interesting idea to make the ioctl()s in the >device driver a no-op instead of not being present or generating its own >errnos. That way folks can turn the knob, feel better, but not see any >change since "it doesn't get any better than this". I like

Re: Interrupt affinity cannot be set on Mellanox card

On Nov 17, 2017, at 4:45 PM, Alan Altmark wrote: > It might, however, be an interesting idea to make the ioctl()s in the > device driver a no-op instead of not being present or generating its own > errnos. That way folks can turn the knob, feel better, but not see any > change since "it doesn't

Re: Interrupt affinity cannot be set on Mellanox card

On Friday, 11/17/2017 at 11:53 GMT, Sebastian Ott wrote: > It's a performance tuning knob - a mechanism for device drivers (and > administrators) to specifiy that they expect an interrupt on the CPU that they > used for IO submission. > > Since all interrupts are

Re: Interrupt affinity cannot be set on Mellanox card

On Fri, 17 Nov 2017, Timothy Sipples wrote: > Sebastian Ott wrote: > >Setting irq affinity is currently not supported on s390. > > This platform's I/O architecture is unique. What are the real-world issues > associated with not being able to set this parameter on this platform? And > what are the

Re: Interrupt affinity cannot be set on Mellanox card

Sebastian Ott wrote: >Setting irq affinity is currently not supported on s390. This platform's I/O architecture is unique. What are the real-world issues associated with not being able to set this parameter on this platform? And what are the suggested mitigations?

Re: Interrupt affinity cannot be set on Mellanox card

> Recently we're testing Mellanox 10GbE performance with Ubuntu 17.04 s390x > on z13. During the test, we found that interrupt affinity cannot be set > like other platforms. I/O-related hacks for other platforms are unlikely to work in the same way on this hardware; the underlying I/O subsystem

Re: Interrupt affinity cannot be set on Mellanox card

Thanks much for the clarification. -James On Thu, Nov 16, 2017 at 5:20 PM, Sebastian Ott wrote: > James, > > On Thu, 16 Nov 2017, Jingmin Zhai wrote: > > Recently we're testing Mellanox 10GbE performance with Ubuntu 17.04 s390x > > on z13. During the test, we found

Re: Interrupt affinity cannot be set on Mellanox card

James, On Thu, 16 Nov 2017, Jingmin Zhai wrote: > Recently we're testing Mellanox 10GbE performance with Ubuntu 17.04 s390x > on z13. During the test, we found that interrupt affinity cannot be set > like other platforms. > > We stopped the 'irqbalance' service first, then > > echo >

Interrupt affinity cannot be set on Mellanox card

Recently we're testing Mellanox 10GbE performance with Ubuntu 17.04 s390x on z13. During the test, we found that interrupt affinity cannot be set like other platforms. We stopped the 'irqbalance' service first, then echo > /proc/irq//smp_affinity But get "echo: write error: Input/output