[RFC V3 PATCH 03/26] net/netpolicy: get device queue irq information

2016-09-12 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Net policy needs to know device information. Currently, it's enough to only get irq information of rx and tx queues. This patch introduces ndo ops to do so, not ethtool ops. Because there are already several ways to get irq information in userspace

[RFC V3 PATCH 10/26] net/netpolicy: add three new NET policies

2016-09-12 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Introduce three NET policies CPU policy: configure for higher throughput and lower CPU% (power saving). BULK policy: configure for highest throughput. LATENCY policy: configure for lowest latency. Signed-off-by: Kan Liang <kan.li...@intel.com>

[RFC V3 PATCH 01/26] net: introduce NET policy

2016-09-12 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> This patch introduce NET policy subsystem. If proc is supported in the system, it creates netpolicy node in proc system. Signed-off-by: Kan Liang <kan.li...@intel.com> --- include/linux/netdevice.h | 7 +++ include/net/net_namespace.h |

[RFC V3 PATCH 04/26] net/netpolicy: get CPU information

2016-09-12 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Net policy also needs to know CPU information. Currently, online CPU count is enough. Signed-off-by: Kan Liang <kan.li...@intel.com> --- net/core/netpolicy.c | 5 + 1 file changed, 5 insertions(+) diff --git a/net/core/netpolicy.

[RFC V3 PATCH 09/26] net/netpolicy: set NET policy by policy name

2016-09-12 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> User can write policy name to /proc/net/netpolicy/$DEV/policy to enable net policy for specific device. When the policy is enabled, the subsystem automatically disables IRQ balance and set IRQ affinity. The object list is also generated accor

[RFC V3 PATCH 14/26] net/netpolicy: handle channel changes

2016-09-12 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> User can uses ethtool to set the channel number. This patch handles the channel changes by rebuilding the object list. Signed-off-by: Kan Liang <kan.li...@intel.com> --- include/linux/netpolicy.h | 8 net/core/ethtool.c| 8 +

[RFC V3 PATCH 11/26] net/netpolicy: add MIX policy

2016-09-12 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> MIX policy is combine of other policies. It allows different queue has different policy. If MIX policy is applied, /proc/net/netpolicy/$DEV/policy shows per queue policy. Usually, the workloads requires either high throughput or low latency. So for c

[RFC V3 PATCH 12/26] net/netpolicy: NET device hotplug

2016-09-12 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Support NET device up/down/namechange in the NET policy code. Signed-off-by: Kan Liang <kan.li...@intel.com> --- net/core/netpolicy.c | 66 +--- 1 file changed, 58 insertions(+), 8 deletions(-)

[RFC V3 PATCH 17/26] net/netpolicy: introduce netpolicy_pick_queue

2016-09-12 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> To achieve better network performance, the key step is to distribute the packets to dedicated queues according to policy and system run time status. This patch provides an interface which can return the proper dedicated queue for socke

[RFC V3 PATCH 13/26] net/netpolicy: support CPU hotplug

2016-09-12 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> For CPU hotplug, the NET policy subsystem will rebuild the sys map and object list. Signed-off-by: Kan Liang <kan.li...@intel.com> --- net/core/netpolicy.c | 80 1 file changed, 80 inserti

[RFC V3 PATCH 15/26] net/netpolicy: implement netpolicy register

2016-09-12 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> The socket/task can only be benefited when it register itself with specific policy. If it's the first time to register, a record will be created and inserted into RCU hash table. The record includes ptr, policy and object information. ptr is the socket/

[RFC V3 PATCH 18/26] net/netpolicy: set tx queues according to policy

2016-09-12 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> When the device tries to transmit a packet, netdev_pick_tx is called to find the available tx queues. If the net policy is applied, it picks up the assigned tx queue from net policy subsystem, and redirect the traffic to the assigned queue. Sign

[RFC V3 PATCH 19/26] net/netpolicy: tc bpf extension to pick Tx queue

2016-09-12 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> This patch extends the netpolicy to support tc bpf when selecting Tx queue. It implements a bpf classifier for clsact qdisc. The classifier will pick up the proper queue from net policy subsystem. This queue selection from tc is not compatible with X

[RFC V3 PATCH 20/26] net/netpolicy: set Rx queues according to policy

2016-09-12 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> For setting Rx queues, this patch configure Rx network flow classification rules to redirect the packets to the assigned queue. Since we may not get all the information required for rule until the first packet arrived, it will add the rule after r

[RFC V3 PATCH 08/26] net/netpolicy: introduce NET policy object

2016-09-12 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> This patch introduces the concept of NET policy object and policy object list. The NET policy object is the instance of CPU/queue mapping. The object can be shared between different tasks/sockets. So besides CPU and queue information, the objec

[RFC V3 PATCH 02/26] net/netpolicy: init NET policy

2016-09-12 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> This patch tries to initialize NET policy for all the devices in the system. However, not all device drivers have NET policy support. For those drivers who does not have NET policy support, the node will not be showed in /proc/net/netpolicy/. The

[RFC V3 PATCH 22/26] net/netpolicy: set per task policy by proc

2016-09-12 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Users may not want to change the source code to add per task net polic support. Or they may want to change a running task's net policy. prctl does not work for both cases. This patch adds an interface in /proc, which can be used to set and retrieve

[RFC V3 PATCH 00/26] Kernel NET policy

2016-09-12 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> It is a big challenge to get good network performance. First, the network performance is not good with default system settings. Second, it is too difficult to do automatic tuning for all possible workloads, since workloads have different requirements

[RFC V3 PATCH 07/26] net/netpolicy: enable and disable NET policy

2016-09-12 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> This patch introduces functions to enable and disable NET policy. For enabling, it collects device and CPU information and setup CPU/queue mapping. Also, for some drivers like i40e driver, it will get better performance if setting IRQ affinity. This

[RFC V3 PATCH 06/26] net/netpolicy: set and remove IRQ affinity

2016-09-12 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> This patches introduces functions to set and remove IRQ affinity according to cpu and queue mapping. The functions will not record the previous affinity status. After a set/remove cycles, it will set the affinity on all online CPU with IRQ balance en

[RFC V3 PATCH 05/26] net/netpolicy: create CPU and queue mapping

2016-09-12 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Current implementation forces CPU and queue 1:1 mapping. This patch introduces the function netpolicy_update_sys_map to create this mapping. The result is stored in netpolicy_sys_info. If the CPU count and queue count are different, the remaining CPUs/

[RFC V3 PATCH 26/26] Documentation/networking: Document NET policy

2016-09-12 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Signed-off-by: Kan Liang <kan.li...@intel.com> --- Documentation/networking/netpolicy.txt | 157 + 1 file changed, 157 insertions(+) create mode 100644 Documentation/networking/netpolicy.txt diff --git a/D

[RFC V3 PATCH 21/26] net/netpolicy: introduce per task net policy

2016-09-12 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Usually, application as a whole has specific requirement. Applying the net policy to all sockets one by one in the application is too complex. This patch introduces per task net policy to address this case. Once the per task net policy is applie

[RFC V3 PATCH 25/26] net/netpolicy: limit the total record number

2016-09-12 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> NET policy can not fulfill users request without limit, because of the security consideration and device limitation. For security consideration, the attacker may fake millions of per task/socket request to crash the system. For device limitation, th

[RFC V3 PATCH 16/26] net/netpolicy: introduce per socket netpolicy

2016-09-12 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> The network socket is the most basic unit which control the network traffic. A socket option is needed for user to set their own policy on socket to improve the network performance. There is no existing SOCKET options which can be reused. For socket o

[RFC V3 PATCH 24/26] net/netpolicy: optimize for queue pair

2016-09-12 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Some drivers like i40e driver does not support separate Tx and Rx queues as channels. Using Rx queue to stand for the channels, if queue_pair is set by driver. Signed-off-by: Kan Liang <kan.li...@intel.com> --- include/linux/netpolicy.h | 1

[RFC V3 PATCH 23/26] net/netpolicy: fast path for finding the queues

2016-09-12 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Current implementation searches the hash table to get assigned object for each transmit/receive packet. It's not necessory, because the assigned object usually remain unchanged. This patch store the assigned queue to speed up the searching process. But

[RFC V2 PATCH 04/25] net/netpolicy: get CPU information

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Net policy also needs to know CPU information. Currently, online CPU number is enough. Signed-off-by: Kan Liang <kan.li...@intel.com> --- net/core/netpolicy.c | 5 + 1 file changed, 5 insertions(+) diff --git a/net/core/netpolicy.

[RFC V2 PATCH 06/25] net/netpolicy: set and remove IRQ affinity

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> This patches introduces functions to set and remove IRQ affinity according to cpu and queue mapping. The functions will not record the previous affinity status. After a set/remove cycles, it will set the affinity on all online CPU with IRQ balance en

[RFC V2 PATCH 01/25] net: introduce NET policy

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> This patch introduce NET policy subsystem. If proc is supported in the system, it creates netpolicy node in proc system. Signed-off-by: Kan Liang <kan.li...@intel.com> --- include/linux/netdevice.h | 7 +++ include/net/net_namespace.h |

[RFC V2 PATCH 02/25] net/netpolicy: init NET policy

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> This patch tries to initialize NET policy for all the devices in the system. However, not all device drivers have NET policy support. For those drivers who does not have NET policy support, the node will not be showed in /proc/net/netpolicy/. The

[RFC V2 PATCH 00/25] Kernel NET policy

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> (re-send to correct system time issue. Sorry for any inconvenience.) It is a big challenge to get good network performance. First, the network performance is not good with default system settings. Second, it is too difficult to do automatic tuning f

[RFC V2 PATCH 05/25] net/netpolicy: create CPU and queue mapping

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Current implementation forces CPU and queue 1:1 mapping. This patch introduces the function netpolicy_update_sys_map to create this mapping. The result is stored in netpolicy_sys_info. If the CPU count and queue count are different, the remaining CPUs/

[RFC V2 PATCH 09/25] net/netpolicy: set NET policy by policy name

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> User can write policy name to /proc/net/netpolicy/$DEV/policy to enable net policy for specific device. When the policy is enabled, the subsystem automatically disables IRQ balance and set IRQ affinity. The object list is also generated accor

[RFC V2 PATCH 08/25] net/netpolicy: introduce NET policy object

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> This patch introduces the concept of NET policy object and policy object list. The NET policy object is the instance of CPU/queue mapping. The object can be shared between different tasks/sockets. So besides CPU and queue information, the objec

[RFC V2 PATCH 12/25] net/netpolicy: NET device hotplug

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Support NET device up/down/namechange in the NET policy code. Signed-off-by: Kan Liang <kan.li...@intel.com> --- net/core/netpolicy.c | 66 +--- 1 file changed, 58 insertions(+), 8 deletions(-)

[RFC V2 PATCH 10/25] net/netpolicy: add three new NET policies

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Introduce three NET policies CPU policy: configure for higher throughput and lower CPU% (power saving). BULK policy: configure for highest throughput. LATENCY policy: configure for lowest latency. Signed-off-by: Kan Liang <kan.li...@intel.com>

[RFC V2 PATCH 07/25] net/netpolicy: enable and disable NET policy

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> This patch introduces functions to enable and disable NET policy. For enabling, it collects device and CPU information, setup CPU/queue mapping, and set IRQ affinity accordingly. For disabling, it removes the IRQ affinity and mapping information. n

[RFC V2 PATCH 15/25] net/netpolicy: implement netpolicy register

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> The socket/task can only be benefited when it register itself with specific policy. If it's the first time to register, a record will be created and inserted into RCU hash table. The record includes ptr, policy and object information. ptr is the socket/

[RFC V2 PATCH 13/25] net/netpolicy: support CPU hotplug

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> For CPU hotplug, the NET policy subsystem will rebuild the sys map and object list. Signed-off-by: Kan Liang <kan.li...@intel.com> --- net/core/netpolicy.c | 76 1 file changed, 76 inserti

[RFC V2 PATCH 11/25] net/netpolicy: add MIX policy

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> MIX policy is combine of other policies. It allows different queue has different policy. If MIX policy is applied, /proc/net/netpolicy/$DEV/policy shows per queue policy. Usually, the workloads requires either high throughput or low latency. So for c

[RFC V2 PATCH 18/25] net/netpolicy: set Tx queues according to policy

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> When the device tries to transmit a packet, netdev_pick_tx is called to find the available Tx queues. If the net policy is applied, it picks up the assigned Tx queue from net policy subsystem, and redirect the traffic to the assigned queue. Sign

[RFC V2 PATCH 16/25] net/netpolicy: introduce per socket netpolicy

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> The network socket is the most basic unit which control the network traffic. This patch introduces a new socket option SO_NETPOLICY to set/get net policy for socket. so that the application can set its own policy on socket to improve the network perfo

[RFC V2 PATCH 21/25] net/netpolicy: set per task policy by proc

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Users may not want to change the source code to add per task net polic support. Or they may want to change a running task's net policy. prctl does not work for both cases. This patch adds an interface in /proc, which can be used to set and retrieve

[RFC V2 PATCH 14/25] net/netpolicy: handle channel changes

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> User can uses ethtool to set the channel number. This patch handles the channel changes by rebuilding the object list. Signed-off-by: Kan Liang <kan.li...@intel.com> --- include/linux/netpolicy.h | 8 net/core/ethtool.c| 8 +

[RFC V2 PATCH 19/25] net/netpolicy: set Rx queues according to policy

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> For setting Rx queues, this patch configure Rx network flow classification rules to redirect the packets to the assigned queue. Since we may not get all the information required for rule until the first packet arrived, it will add the rule after r

[RFC V2 PATCH 17/25] net/netpolicy: introduce netpolicy_pick_queue

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> To achieve better network performance, the key step is to distribute the packets to dedicated queues according to policy and system run time status. This patch provides an interface which can return the proper dedicated queue for socke

[RFC V2 PATCH 25/25] Documentation/networking: Document NET policy

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Signed-off-by: Kan Liang <kan.li...@intel.com> --- Documentation/networking/netpolicy.txt | 157 + 1 file changed, 157 insertions(+) create mode 100644 Documentation/networking/netpolicy.txt diff --git a/D

[RFC V2 PATCH 24/25] net/netpolicy: limit the total record number

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> NET policy can not fulfill users request without limit, because of the security consideration and device limitation. For security consideration, the attacker may fake millions of per task/socket request to crash the system. For device limitation, th

[RFC V2 PATCH 23/25] net/netpolicy: optimize for queue pair

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Some drivers like i40e driver does not support separate Tx and Rx queues as channels. Using Rx queue to stand for the channels, if queue_pair is set by driver. Signed-off-by: Kan Liang <kan.li...@intel.com> --- include/linux/netpolicy.h | 1

[RFC V2 PATCH 22/25] net/netpolicy: fast path for finding the queues

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Current implementation searches the hash table to get assigned object for each transmit/receive packet. It's not necessory, because the assigned object usually remain unchanged. This patch store the assigned queue to speed up the searching process. But

[RFC V2 PATCH 20/25] net/netpolicy: introduce per task net policy

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Usually, application as a whole has specific requirement. Applying the net policy to all sockets one by one in the application is too complex. This patch introduces per task net policy to address this case. Once the per task net policy is applie

[RFC V2 PATCH 03/25] net/netpolicy: get device queue irq information

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Net policy needs to know device information. Currently, it's enough to only get irq information of rx and tx queues. This patch introduces ndo ops to do so, not ethtool ops. Because there are already several ways to get irq information in userspace

[RFC V2 PATCH 04/25] net/netpolicy: get CPU information

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Net policy also needs to know CPU information. Currently, online CPU number is enough. Signed-off-by: Kan Liang <kan.li...@intel.com> --- net/core/netpolicy.c | 5 + 1 file changed, 5 insertions(+) diff --git a/net/core/netpolicy.

[RFC V2 PATCH 01/25] net: introduce NET policy

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> This patch introduce NET policy subsystem. If proc is supported in the system, it creates netpolicy node in proc system. Signed-off-by: Kan Liang <kan.li...@intel.com> --- include/linux/netdevice.h | 7 +++ include/net/net_namespace.h |

[RFC V2 PATCH 11/25] net/netpolicy: add MIX policy

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> MIX policy is combine of other policies. It allows different queue has different policy. If MIX policy is applied, /proc/net/netpolicy/$DEV/policy shows per queue policy. Usually, the workloads requires either high throughput or low latency. So for c

[RFC V2 PATCH 03/25] net/netpolicy: get device queue irq information

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Net policy needs to know device information. Currently, it's enough to only get irq information of rx and tx queues. This patch introduces ndo ops to do so, not ethtool ops. Because there are already several ways to get irq information in userspace

[RFC V2 PATCH 12/25] net/netpolicy: NET device hotplug

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Support NET device up/down/namechange in the NET policy code. Signed-off-by: Kan Liang <kan.li...@intel.com> --- net/core/netpolicy.c | 66 +--- 1 file changed, 58 insertions(+), 8 deletions(-)

[RFC V2 PATCH 02/25] net/netpolicy: init NET policy

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> This patch tries to initialize NET policy for all the devices in the system. However, not all device drivers have NET policy support. For those drivers who does not have NET policy support, the node will not be showed in /proc/net/netpolicy/. The

[RFC V2 PATCH 13/25] net/netpolicy: support CPU hotplug

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> For CPU hotplug, the NET policy subsystem will rebuild the sys map and object list. Signed-off-by: Kan Liang <kan.li...@intel.com> --- net/core/netpolicy.c | 76 1 file changed, 76 inserti

[RFC V2 PATCH 05/25] net/netpolicy: create CPU and queue mapping

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Current implementation forces CPU and queue 1:1 mapping. This patch introduces the function netpolicy_update_sys_map to create this mapping. The result is stored in netpolicy_sys_info. If the CPU count and queue count are different, the remaining CPUs/

[RFC V2 PATCH 00/25] Kernel NET policy

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> It is a big challenge to get good network performance. First, the network performance is not good with default system settings. Second, it is too difficult to do automatic tuning for all possible workloads, since workloads have different requirements

[RFC V2 PATCH 19/25] net/netpolicy: set Rx queues according to policy

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> For setting Rx queues, this patch configure Rx network flow classification rules to redirect the packets to the assigned queue. Since we may not get all the information required for rule until the first packet arrived, it will add the rule after r

[RFC V2 PATCH 15/25] net/netpolicy: implement netpolicy register

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> The socket/task can only be benefited when it register itself with specific policy. If it's the first time to register, a record will be created and inserted into RCU hash table. The record includes ptr, policy and object information. ptr is the socket/

[RFC V2 PATCH 18/25] net/netpolicy: set Tx queues according to policy

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> When the device tries to transmit a packet, netdev_pick_tx is called to find the available Tx queues. If the net policy is applied, it picks up the assigned Tx queue from net policy subsystem, and redirect the traffic to the assigned queue. Sign

[RFC V2 PATCH 09/25] net/netpolicy: set NET policy by policy name

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> User can write policy name to /proc/net/netpolicy/$DEV/policy to enable net policy for specific device. When the policy is enabled, the subsystem automatically disables IRQ balance and set IRQ affinity. The object list is also generated accor

[RFC V2 PATCH 16/25] net/netpolicy: introduce per socket netpolicy

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> The network socket is the most basic unit which control the network traffic. This patch introduces a new socket option SO_NETPOLICY to set/get net policy for socket. so that the application can set its own policy on socket to improve the network perfo

[RFC V2 PATCH 17/25] net/netpolicy: introduce netpolicy_pick_queue

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> To achieve better network performance, the key step is to distribute the packets to dedicated queues according to policy and system run time status. This patch provides an interface which can return the proper dedicated queue for socke

[RFC V2 PATCH 14/25] net/netpolicy: handle channel changes

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> User can uses ethtool to set the channel number. This patch handles the channel changes by rebuilding the object list. Signed-off-by: Kan Liang <kan.li...@intel.com> --- include/linux/netpolicy.h | 8 net/core/ethtool.c| 8 +

[RFC V2 PATCH 07/25] net/netpolicy: enable and disable NET policy

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> This patch introduces functions to enable and disable NET policy. For enabling, it collects device and CPU information, setup CPU/queue mapping, and set IRQ affinity accordingly. For disabling, it removes the IRQ affinity and mapping information. n

[RFC V2 PATCH 21/25] net/netpolicy: set per task policy by proc

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Users may not want to change the source code to add per task net polic support. Or they may want to change a running task's net policy. prctl does not work for both cases. This patch adds an interface in /proc, which can be used to set and retrieve

[RFC V2 PATCH 20/25] net/netpolicy: introduce per task net policy

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Usually, application as a whole has specific requirement. Applying the net policy to all sockets one by one in the application is too complex. This patch introduces per task net policy to address this case. Once the per task net policy is applie

[RFC V2 PATCH 24/25] net/netpolicy: limit the total record number

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> NET policy can not fulfill users request without limit, because of the security consideration and device limitation. For security consideration, the attacker may fake millions of per task/socket request to crash the system. For device limitation, th

[RFC V2 PATCH 22/25] net/netpolicy: fast path for finding the queues

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Current implementation searches the hash table to get assigned object for each transmit/receive packet. It's not necessory, because the assigned object usually remain unchanged. This patch store the assigned queue to speed up the searching process. But

[RFC V2 PATCH 06/25] net/netpolicy: set and remove IRQ affinity

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> This patches introduces functions to set and remove IRQ affinity according to cpu and queue mapping. The functions will not record the previous affinity status. After a set/remove cycles, it will set the affinity on all online CPU with IRQ balance en

[RFC V2 PATCH 23/25] net/netpolicy: optimize for queue pair

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Some drivers like i40e driver does not support separate Tx and Rx queues as channels. Using Rx queue to stand for the channels, if queue_pair is set by driver. Signed-off-by: Kan Liang <kan.li...@intel.com> --- include/linux/netpolicy.h | 1

[RFC V2 PATCH 10/25] net/netpolicy: add three new NET policies

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Introduce three NET policies CPU policy: configure for higher throughput and lower CPU% (power saving). BULK policy: configure for highest throughput. LATENCY policy: configure for lowest latency. Signed-off-by: Kan Liang <kan.li...@intel.com>

[RFC V2 PATCH 08/25] net/netpolicy: introduce NET policy object

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> This patch introduces the concept of NET policy object and policy object list. The NET policy object is the instance of CPU/queue mapping. The object can be shared between different tasks/sockets. So besides CPU and queue information, the objec

[RFC V2 PATCH 25/25] Documentation/networking: Document NET policy

2016-08-04 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Signed-off-by: Kan Liang <kan.li...@intel.com> --- Documentation/networking/netpolicy.txt | 157 + 1 file changed, 157 insertions(+) create mode 100644 Documentation/networking/netpolicy.txt diff --git a/D

[RFC PATCH 01/30] net: introduce NET policy

2016-07-18 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> This patch introduce NET policy subsystem. If proc is supported in the system, it creates netpolicy node in proc system. Signed-off-by: Kan Liang <kan.li...@intel.com> --- include/linux/netdevice.h | 7 +++ include/net/net_namespace.h |

[RFC PATCH 05/30] i40e/netpolicy: implement ndo_get_irq_info

2016-07-18 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Implement ndo_get_irq_info in i40e driver to get irq information of rx and tx queues. Signed-off-by: Kan Liang <kan.li...@intel.com> --- drivers/net/ethernet/intel/i40e/i40e_main.c | 40 + 1 file changed, 40 inserti

[RFC PATCH 02/30] net/netpolicy: init NET policy

2016-07-18 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> This patch tries to initialize NET policy for all the devices in the system. However, not all device drivers have NET policy support. For those drivers who does not have NET policy support, the node will not be showed in /proc/net/netpolicy/. The

[RFC PATCH 09/30] net/netpolicy: enable and disable net policy

2016-07-18 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> This patch introduces functions to enable and disable net policy. For enabling, it collects device and cpu information, setup cpu/queue mapping, and set irq affinity accordingly. For disabling, it removes the irq affinity and mapping information. n

[RFC PATCH 08/30] net/netpolicy: set and remove irq affinity

2016-07-18 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> This patches introduces functions to set and remove irq affinity according to cpu and queue mapping. The functions will not record the previous affinity status. After a set/remove cycles, it will set the affinity on all online cpu with irq balance en

[RFC PATCH 07/30] net/netpolicy: create CPU and queue mapping

2016-07-18 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Current implementation forces CPU and queue 1:1 mapping. This patch introduces the function netpolicy_update_sys_map to create this mapping. The result is stored in netpolicy_sys_info. If the CPU count and queue count are different, the remaining CPUs/

[RFC PATCH 06/30] net/netpolicy: get CPU information

2016-07-18 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Net policy also needs to know CPU information. Currently, online CPU number is enough. Signed-off-by: Kan Liang <kan.li...@intel.com> --- net/core/netpolicy.c | 5 + 1 file changed, 5 insertions(+) diff --git a/net/core/netpolicy.

[RFC PATCH 03/30] i40e/netpolicy: Implement ndo_netpolicy_init

2016-07-18 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Support ndo_netpolicy_init in i40e driver. For i40e driver, there is no extra initialization work to do. It only needs to update the available policy bitmap. policy_param will be filled according to different policies later. Signed-off-by: Kan Liang &

[RFC PATCH 16/30] net/netpolicy: net device hotplug

2016-07-18 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Support net device up/down/namechange in the netpolicy code. Signed-off-by: Kan Liang <kan.li...@intel.com> --- net/core/netpolicy.c | 66 +--- 1 file changed, 58 insertions(+), 8 deletions(-)

[RFC PATCH 04/30] net/netpolicy: get driver information

2016-07-18 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Net policy needs to know device information. Currently, it's enough to only get irq information of rx and tx queues. This patch introduces ndo_get_irq_info to do so. Signed-off-by: Kan Liang <kan.li...@intel.com> --- include/linux/netd

[RFC PATCH 13/30] i40e/netpolicy: add three new net policies

2016-07-18 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Introduce three net policies for i40e driver. CPU policy: configure for higher throughput and lower CPU%. BULK policy: configure for highest throughput. LATENCY policy: configure for lowest latency. Lots of tests are done for net policy on pla

[RFC PATCH 00/30] Kernel NET policy

2016-07-18 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> It is a big challenge to get good network performance. First, the network performance is not good with default system settings. Second, it is too difficult to do automatic tuning for all possible workloads, since workloads have different requirements

[RFC PATCH 11/30] net/netpolicy: set net policy by policy name

2016-07-18 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> User can write policy name to /proc/net/netpolicy/$DEV/policy to enable net policy for specific device. When the policy is enabled, the module automatically disables irq balance and set irq affinity. The object list is also generated accordingly. n

[RFC PATCH 17/30] net/netpolicy: support CPU hotplug

2016-07-18 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> For CPU hotplug, the net policy module will rebuild the sys map and object list. Signed-off-by: Kan Liang <kan.li...@intel.com> --- net/core/netpolicy.c | 78 1 file changed, 78 insertions(+)

[RFC PATCH 15/30] i40e/netpolicy: add MIX policy support

2016-07-18 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Enable i40e MIX policy support. Based on the test, the MIX policy has better performance if increasing rx interrupt moderation a little bit. For evaluating the MIX policy performance, mixed workloads are tested. The mixed workloads are combi

[RFC PATCH 18/30] net/netpolicy: handle channel changes

2016-07-18 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> User can uses ethtool to set the channel number. This patch handles the channel changes by rebuilding the object list. Signed-off-by: Kan Liang <kan.li...@intel.com> --- include/linux/netpolicy.h | 8 net/core/ethtool.c| 8 +

[RFC PATCH 22/30] net/netpolicy: set tx queues according to policy

2016-07-18 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> When the device tries to transmit a buffer, netdev_pick_tx is called to find the available tx queues. This patch checks the per socket net policy of the binding socket of the buffer. If net policy is set, it picks up the assigned tx queue from net

[RFC PATCH 26/30] net/netpolicy: set per task policy by proc

2016-07-18 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> Users may not want to change the source code to add per task net polic support. Or they may want to change a running task's net policy. prctl does not work for both cases. This patch adds an interface in /proc, which can be used to set and retrieve

[RFC PATCH 14/30] net/netpolicy: add MIX policy

2016-07-18 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> MIX policy is combine of other policies. It allows different queue has different policy. If MIX policy is applied, /proc/net/netpolicy/$DEV/policy shows per queue policy. Usually, the workloads requires either high throughput or low latency. So for c

[RFC PATCH 10/30] net/netpolicy: introduce netpolicy object

2016-07-18 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> This patch introduces the concept of netpolicy object and policy object list. The netpolicy object is the instance of CPU/queue mapping. The object can be shared between different tasks/sockets. So besides CPU and queue information, the object also mai

[RFC PATCH 24/30] net/netpolicy: set rx queues according to policy

2016-07-18 Thread kan . liang
From: Kan Liang <kan.li...@intel.com> For setting rx queues, this patch add rules for Flow Director filters. Since we may not get all the information required for rule until the first package arrived, it will add the rule after recvmsg. The first several packages may not use the assigned

  1   2   >