Re: Fwd: Fwd: Re: [PATCH net-next 00/15] net/smc: Shared Memory Communications - RDMA

2016-07-12 Thread Benjamin Poirier 
On 2016/07/12 11:08, Benjamin Poirier wrote:
> On 2016/07/06 17:29, Ursula Braun wrote:
> > Dave,
> > 
> > we still like to see SMC-R included into a future Linux-kernel. After
> > answering your first 2 questions, there is no longer a response. What should
> > we do next?
> > - Still wait for an answer from you?
> > - Resend the same whole SMC-R patch series, this time with the cover letter
> > adapted to your requested changes?
> 
> ^^^ I would suggest to send v2 of the patch series with the changes
> that were requested.
> 
> > - Put the SMC-R development on hold, and concentrate on another
> > s390-specific SMC-solution first (not RDMA-based), that makes use of the
> > SMC-socket family as well.
> > - Anything else?
> > 

... and I would suggest to trim my emails next time. Sorry.

Re: Fwd: Fwd: Re: [PATCH net-next 00/15] net/smc: Shared Memory Communications - RDMA

2016-07-12 Thread Benjamin Poirier 
On 2016/07/06 17:29, Ursula Braun wrote:
> Dave,
> 
> we still like to see SMC-R included into a future Linux-kernel. After
> answering your first 2 questions, there is no longer a response. What should
> we do next?
> - Still wait for an answer from you?
> - Resend the same whole SMC-R patch series, this time with the cover letter
> adapted to your requested changes?

^^^ I would suggest to send v2 of the patch series with the changes
that were requested.

> - Put the SMC-R development on hold, and concentrate on another
> s390-specific SMC-solution first (not RDMA-based), that makes use of the
> SMC-socket family as well.
> - Anything else?
> 
> Kind regards, Ursula
> 
> -------- Forwarded Message --------
> Subject: Fwd: Re: [PATCH net-next 00/15] net/smc: Shared Memory
> Communications - RDMA
> Date: Tue, 21 Jun 2016 16:02:59 +0200
> From: Ursula Braun <ubr...@linux.vnet.ibm.com>
> To: da...@davemloft.net
> CC: netdev@vger.kernel.org, linux-s...@vger.kernel.org,
> schwidef...@de.ibm.com, heiko.carst...@de.ibm.com, utz.bac...@de.ibm.com
> 
> Dave,
> 
> the SMC-R patches submitted 2016-06-03 show up in state "Changes
> Requested" on patchwork:
> https://patchwork.ozlabs.org/project/netdev/list/?submitter=2266=*=1
> 
> You had requested a change of the SMC-R description in the cover letter.
> We came up with the response below. Do you need anything else from us?
> 
> Kind regards,
> Ursula Braun
> 
>  Forwarded Message 
> Subject: Re: [PATCH net-next 00/15] net/smc: Shared Memory
> Communications - RDMA
> Date: Thu,  9 Jun 2016 17:36:28 +0200
> From: Ursula Braun <ubr...@linux.vnet.ibm.com>
> To: da...@davemloft.net
> CC: netdev@vger.kernel.org, linux-s...@vger.kernel.org,
> schwidef...@de.ibm.com, heiko.carst...@de.ibm.com
> 
> On Tue, 2016-06-07 at 15:07 -0700, David Miller wrote:
> > In case my previous reply wasn't clear enough, I require that you provide
> > a more accurate description of what the implications of this feature are.
> > 
> > Namely, that important _CORE_ networking features are completely bypassed
> > and unusable when SMC applies to a connection.
> > 
> > Specifically, all packet shaping, filtering, traffic inspection, and
> > flow management facilitites in the kernel will not be able to see nor
> > act upon the data flow of these TCP connections once established.
> > 
> > It is always important, and in my opinion required, to list the
> > negative aspects of your change and not just the "wow, amazing"
> > positive aspects.
> > 
> > Thanks.
> > 
> > 
> Correct, the SMC-R data stream bypasses TCP and thus cannot enjoy its
> features. This is the price for leveraging the TCP application ecosystem
> and reducing CPU load.
> 
> When a load balancer allows the TCP handshake to take place between a
> worker node and the TCP client, RDMA will be used between these two
> nodes. So anything based on TCP connection establishment (including a
> firewall) can apply to SMC-R, too. To be clear -- yes, the data flow
> later on is not subject to these features anymore.  At least VLAN
> isolation from the TCP part can be leveraged for RDMA traffic. From our
> experience, discussions, etc., that tradeoff seems acceptable in a
> classical data center environment.
> 
> Improving our cover letter would result in the following new
> introductory motivation part at the beginning and a slightly modified
> list of
> planned enhancements at the end:
> 
> On Fri, 2016-06-03 at 17:26 +0200, Ursula Braun wrote:
> 
> > These patches are the initial part of the implementation of the
> > "Shared Memory Communications-RDMA" (SMC-R) protocol. The protocol is
> > defined in RFC7609 [1]. It allows transformation of TCP connections
> > using the "Remote Direct Memory Access over Converged Ethernet" (RoCE)
> > feature of specific communication hardware for data center environments.
> > 
> > SMC-R inherits TCP qualities such as reliable connections, host-based
> > firewall packet filtering (on connection establishment) and unmodified
> > application of communication encryption such as TLS (transport layer
> > security) or SSL (secure sockets layer). It is transparent to most existing
> > TCP connection load balancers that are commonly used in the enterprise data
> > center environment for multi-tier application workloads.
> > 
> > Being designed for the data center network switched fabric environment, it
> > does not need congestion control and thus reaches line speed right away
> > without having to go through slow start as with TCP. This can be beneficial
> &g

Fwd: Fwd: Re: [PATCH net-next 00/15] net/smc: Shared Memory Communications - RDMA

2016-07-06 Thread Ursula Braun 

Dave,

we still like to see SMC-R included into a future Linux-kernel. After 
answering your first 2 questions, there is no longer a response. What 
should we do next?

- Still wait for an answer from you?
- Resend the same whole SMC-R patch series, this time with the cover 
letter adapted to your requested changes?
- Put the SMC-R development on hold, and concentrate on another 
s390-specific SMC-solution first (not RDMA-based), that makes use of the 
SMC-socket family as well.

- Anything else?

Kind regards, Ursula

 Forwarded Message 
Subject: Fwd: Re: [PATCH net-next 00/15] net/smc: Shared Memory 
Communications - RDMA

Date: Tue, 21 Jun 2016 16:02:59 +0200
From: Ursula Braun <ubr...@linux.vnet.ibm.com>
To: da...@davemloft.net
CC: netdev@vger.kernel.org, linux-s...@vger.kernel.org, 
schwidef...@de.ibm.com, heiko.carst...@de.ibm.com, utz.bac...@de.ibm.com


Dave,

the SMC-R patches submitted 2016-06-03 show up in state "Changes
Requested" on patchwork:
https://patchwork.ozlabs.org/project/netdev/list/?submitter=2266=*=1

You had requested a change of the SMC-R description in the cover letter.
We came up with the response below. Do you need anything else from us?

Kind regards,
Ursula Braun

 Forwarded Message 
Subject: Re: [PATCH net-next 00/15] net/smc: Shared Memory
Communications - RDMA
Date: Thu,  9 Jun 2016 17:36:28 +0200
From: Ursula Braun <ubr...@linux.vnet.ibm.com>
To: da...@davemloft.net
CC: netdev@vger.kernel.org, linux-s...@vger.kernel.org,
schwidef...@de.ibm.com, heiko.carst...@de.ibm.com

On Tue, 2016-06-07 at 15:07 -0700, David Miller wrote:

In case my previous reply wasn't clear enough, I require that you provide
a more accurate description of what the implications of this feature are.

Namely, that important _CORE_ networking features are completely bypassed
and unusable when SMC applies to a connection.

Specifically, all packet shaping, filtering, traffic inspection, and
flow management facilitites in the kernel will not be able to see nor
act upon the data flow of these TCP connections once established.

It is always important, and in my opinion required, to list the
negative aspects of your change and not just the "wow, amazing"
positive aspects.

Thanks.



Correct, the SMC-R data stream bypasses TCP and thus cannot enjoy its
features. This is the price for leveraging the TCP application ecosystem
and reducing CPU load.

When a load balancer allows the TCP handshake to take place between a
worker node and the TCP client, RDMA will be used between these two
nodes. So anything based on TCP connection establishment (including a
firewall) can apply to SMC-R, too. To be clear -- yes, the data flow
later on is not subject to these features anymore.  At least VLAN
isolation from the TCP part can be leveraged for RDMA traffic. From our
experience, discussions, etc., that tradeoff seems acceptable in a
classical data center environment.

Improving our cover letter would result in the following new
introductory motivation part at the beginning and a slightly modified
list of
planned enhancements at the end:

On Fri, 2016-06-03 at 17:26 +0200, Ursula Braun wrote:


These patches are the initial part of the implementation of the
"Shared Memory Communications-RDMA" (SMC-R) protocol. The protocol is
defined in RFC7609 [1]. It allows transformation of TCP connections
using the "Remote Direct Memory Access over Converged Ethernet" (RoCE)
feature of specific communication hardware for data center environments.

SMC-R inherits TCP qualities such as reliable connections, host-based
firewall packet filtering (on connection establishment) and unmodified
application of communication encryption such as TLS (transport layer
security) or SSL (secure sockets layer). It is transparent to most existing
TCP connection load balancers that are commonly used in the enterprise data
center environment for multi-tier application workloads.

Being designed for the data center network switched fabric environment, it
does not need congestion control and thus reaches line speed right away
without having to go through slow start as with TCP. This can be beneficial
for short living flows including request response patterns requiring
reliability. A full SMC-R implementation also provides seamless high
availability and load-balancing demanded by enterprise installations.

SMC-R does not require an RDMA communication manager (RDMA CM). Its use of
RDMA provides CPU savings transparently for unmodified applications.
For instance, when running 10 parallel connections with uperf, we measured
a decrease of 60% in CPU consumption with SMC-R compared to TCP/IP
(with throughput and latency comparable;
measured on x86_64 with the same RoCE card and port).


These patches are the initial part of the implementation of the
"Shared Memory Communications-RDMA" (SMC-R) protocol as defined in
RFC7609 [1].  While SMC-R does not aim to re

Fwd: Re: [PATCH net-next 00/15] net/smc: Shared Memory Communications - RDMA

2016-06-21 Thread Ursula Braun 

Dave,

the SMC-R patches submitted 2016-06-03 show up in state "Changes 
Requested" on patchwork:

https://patchwork.ozlabs.org/project/netdev/list/?submitter=2266=*=1

You had requested a change of the SMC-R description in the cover letter. 
We came up with the response below. Do you need anything else from us?


Kind regards,
Ursula Braun

 Forwarded Message 
Subject: Re: [PATCH net-next 00/15] net/smc: Shared Memory 
Communications - RDMA

Date: Thu,  9 Jun 2016 17:36:28 +0200
From: Ursula Braun 
To: da...@davemloft.net
CC: netdev@vger.kernel.org, linux-s...@vger.kernel.org, 
schwidef...@de.ibm.com, heiko.carst...@de.ibm.com


On Tue, 2016-06-07 at 15:07 -0700, David Miller wrote:

In case my previous reply wasn't clear enough, I require that you provide
a more accurate description of what the implications of this feature are.

Namely, that important _CORE_ networking features are completely bypassed
and unusable when SMC applies to a connection.

Specifically, all packet shaping, filtering, traffic inspection, and
flow management facilitites in the kernel will not be able to see nor
act upon the data flow of these TCP connections once established.

It is always important, and in my opinion required, to list the
negative aspects of your change and not just the "wow, amazing"
positive aspects.

Thanks.



Correct, the SMC-R data stream bypasses TCP and thus cannot enjoy its
features. This is the price for leveraging the TCP application ecosystem
and reducing CPU load.

When a load balancer allows the TCP handshake to take place between a
worker node and the TCP client, RDMA will be used between these two
nodes. So anything based on TCP connection establishment (including a
firewall) can apply to SMC-R, too. To be clear -- yes, the data flow
later on is not subject to these features anymore.  At least VLAN
isolation from the TCP part can be leveraged for RDMA traffic. From our
experience, discussions, etc., that tradeoff seems acceptable in a
classical data center environment.

Improving our cover letter would result in the following new
introductory motivation part at the beginning and a slightly modified 
list of

planned enhancements at the end:

On Fri, 2016-06-03 at 17:26 +0200, Ursula Braun wrote:


These patches are the initial part of the implementation of the
"Shared Memory Communications-RDMA" (SMC-R) protocol. The protocol is
defined in RFC7609 [1]. It allows transformation of TCP connections
using the "Remote Direct Memory Access over Converged Ethernet" (RoCE)
feature of specific communication hardware for data center environments.

SMC-R inherits TCP qualities such as reliable connections, host-based
firewall packet filtering (on connection establishment) and unmodified
application of communication encryption such as TLS (transport layer
security) or SSL (secure sockets layer). It is transparent to most existing
TCP connection load balancers that are commonly used in the enterprise data
center environment for multi-tier application workloads.

Being designed for the data center network switched fabric environment, it
does not need congestion control and thus reaches line speed right away
without having to go through slow start as with TCP. This can be beneficial
for short living flows including request response patterns requiring
reliability. A full SMC-R implementation also provides seamless high
availability and load-balancing demanded by enterprise installations.

SMC-R does not require an RDMA communication manager (RDMA CM). Its use of
RDMA provides CPU savings transparently for unmodified applications.
For instance, when running 10 parallel connections with uperf, we measured
a decrease of 60% in CPU consumption with SMC-R compared to TCP/IP
(with throughput and latency comparable;
measured on x86_64 with the same RoCE card and port).


These patches are the initial part of the implementation of the
"Shared Memory Communications-RDMA" (SMC-R) protocol as defined in
RFC7609 [1].  While SMC-R does not aim to replace TCP,
it taps a wealth of existing data center TCP socket applications
to become more efficient without the need for rewriting them.
SMC-R uses RDMA over Converged Ethernet (RoCE) to save CPU consumption.
For instance, when running 10 parallel connections with uperf, we measured
a decrease of 60% in CPU consumption with SMC-R compared to TCP/IP
(with throughput and latency comparable;
measured on x86_64 with the same RoCE card and port).

SMC-R does not require an RDMA communication manager (RDMA CM).

SMC-R inherits TCP qualities such as reliable connections, host-based
firewall packet filtering (on connection establishment) and unmodified
application of communication encryption such as TLS (transport layer
security) or SSL (secure sockets layer). Since original TCP is used to
establish SMC-R connections, load balancers and packet inspection based
on TCP/IP connection establishment continue to work for SMC-R.

On the other hand