Re: [GROW] is TCP the right layer for BMP session resumption?

[Thomas.Graf]

Hi Tim,

Many thanks for the feedback and input. Much appreciated. My apology for late 
reply.

In section 5 of draft-tppy-bmp-seamless-session
https://tools.ietf.org/html/draft-tppy-bmp-seamless-session-00#section-5

The BMP session lifecycle (not to be confused with TCP session lifecycle) is 
extended with this draft. The BMP session no longer closes with the TCP 
session. Once the TCP TFO session closes, a BMP session timeout counter starts 
counting down. If the TCP TFO session is re-established within this time frame, 
than the BMP session is considered to be "up" during the TCP TFO 
reestablishment time.

The TCP back pressure from BMP monitoring station to the router occurs under 
congestion. Therefore buffering at the monitored router for the BMP session 
occurs during normal BMP session lifetime. With 
draft-tppy-bmp-seamless-session, we make use of this buffer mechanism. 
Different is that it is being used not only in congestion, but also between 
re-establishment of the TCP TFO session (not to be confused with the BMP 
session).

As you pointed out, it is important that BMP message type peer_up and peer_down 
are not being lost during the re-establishment of the TCP TFO session. For that 
very purpose we described in section 5 that buffering must occur for all BMP 
message types which is including BMP peer_up, peer_down, statistics, 
route-mirroring and route-monitoring. I take that as an input to more clearly 
state that in the draft. Jakob made an interesting input that for BMP buffer 
optimization purposes only withdrawals of route-monitoring messages should be 
buffered.

I agree that BMP lacks of a mechanism that the monitoring station can request a 
BMP route-monitoring refresh to the monitored router. I agree as well that if 
that mechanism would be present, than the BMP session lifecycle should be 
re-adjusted to disable/enable initial BMP RIB dump with route-monitoring or 
not. Where I am unsure is wherever it makes sense to implement BMP 
route-refresh within the BMP session protocol or not. Therefore changing BMP to 
be a bi-directional protocol. 

Here I like to get the feedback from the GROW mailing list. If you do see value 
in BMP route-refresh as well, how granular it should be, and wherever it should 
be part of the BMP session or not. 

Best wishes
Thomas

-Original Message-
From: Tim Evens (tievens)  
Sent: Friday, March 12, 2021 10:36 PM
To: Graf Thomas, INI-NET-TCZ-ZH1 ; Jakob Heitz 
(jheitz) ; rainsword.w...@huawei.com; rob...@raszuk.net; 
j...@dataplane.org
Cc: grow@ietf.org
Subject: Re: [GROW] is TCP the right layer for BMP session resumption?

The use of UDP vs TCP is use-case specific.  For example, are you logging and 
don't care if you miss messages or are you maintaining RIB states for 
applications like SDN?   

In terms of accurate logging (ordered regardless of timestamp) and maintaining 
state… TCP is required otherwise we introduce out-of-order and loss recovery 
complexities.  BGP PDU order is required in order to track changes and to 
maintain accurate RIB states.   While a SEQ number in BMP can help to 
re-sequence messages, that puts a lot on every BMP receiver/client. For 
example, BMP receivers will now have to buffer messages and re-sequence them to 
ensure proper ordering when processing.   If buffers are exceeded, what happens 
to those messages and how would the BMP receiver request those missing 
messages/pdus?  Regardless of how, this adds complexity to both the sender and 
receiver.  IMO, this is not addressing the problem of RIB dumps or picking up 
where you left off on reconnect.  

The draft suggests to use TCP_FAST_OPEN, which I believe adds more complexity 
while not solving the other challenges relating to RIB dumps/refreshes.   It 
doesn't address PEER UP handling on reconnect, how to handle peers that change 
or are new during the no-connection time,  and on how to request a peer refresh 
when needed.   It also doesn't address the buffer exhaustion problem on the 
sender (router) side.  IMO, the sender should be configured using buffer sizes 
per receiver and not based on time.   The amount of time is relative to the 
number of updates.  For example, a refresh to update policies will 
flood/exhaust buffers quickly in seconds while normal updates may last for 
minutes without buffer exhaustion.   

There are at least three problems with RIB dumps/reconnects to be solved:

1) Transient reconnects due to network failures, restarts of receivers, etc. 
are resulting in unnecessary INITs, RIB dumps, and PEER UPs.  A PEER UP 
normally means that the receiver invalidates all previous RIB entries as it 
does not know if things were changed/removed during the gap (from last PEER 
UP/DOWN) period.  A RIB dump is expected to refresh the peer RIB upon a PEER 
UP.  IMO, what we need is application level control so the BMP receiver can 
send a control message to the sender to indicate what's needed per-peer.  For 
example, a receiver restart (new 

Re: [GROW] is TCP the right layer for BMP session resumption?

[Tim Evens (tievens)]

The use of UDP vs TCP is use-case specific.  For example, are you logging and 
don't care if you miss messages or are you maintaining RIB states for 
applications like SDN?   

In terms of accurate logging (ordered regardless of timestamp) and maintaining 
state… TCP is required otherwise we introduce out-of-order and loss recovery 
complexities.  BGP PDU order is required in order to track changes and to 
maintain accurate RIB states.   While a SEQ number in BMP can help to 
re-sequence messages, that puts a lot on every BMP receiver/client. For 
example, BMP receivers will now have to buffer messages and re-sequence them to 
ensure proper ordering when processing.   If buffers are exceeded, what happens 
to those messages and how would the BMP receiver request those missing 
messages/pdus?  Regardless of how, this adds complexity to both the sender and 
receiver.  IMO, this is not addressing the problem of RIB dumps or picking up 
where you left off on reconnect.  

The draft suggests to use TCP_FAST_OPEN, which I believe adds more complexity 
while not solving the other challenges relating to RIB dumps/refreshes.   It 
doesn't address PEER UP handling on reconnect, how to handle peers that change 
or are new during the no-connection time,  and on how to request a peer refresh 
when needed.   It also doesn't address the buffer exhaustion problem on the 
sender (router) side.  IMO, the sender should be configured using buffer sizes 
per receiver and not based on time.   The amount of time is relative to the 
number of updates.  For example, a refresh to update policies will 
flood/exhaust buffers quickly in seconds while normal updates may last for 
minutes without buffer exhaustion.   

There are at least three problems with RIB dumps/reconnects to be solved:

1) Transient reconnects due to network failures, restarts of receivers, etc. 
are resulting in unnecessary INITs, RIB dumps, and PEER UPs.  A PEER UP 
normally means that the receiver invalidates all previous RIB entries as it 
does not know if things were changed/removed during the gap (from last PEER 
UP/DOWN) period.  A RIB dump is expected to refresh the peer RIB upon a PEER 
UP.  IMO, what we need is application level control so the BMP receiver can 
send a control message to the sender to indicate what's needed per-peer.  For 
example, a receiver restart (new connection) may require a full refresh of PEER 
UPs and RIB dumps, partial refresh on a subset of peers, only new peers since 
last reconnect time, and/or no refresh at all for any of the given peers.  
Unless a refresh/RIB dump is requested or needed, messages should continue 
where they left off based on buffer allocations (e.g., offsets or similar).  
IMO, the fast reconnect does not address the set of peers, especially 
considering the peers that changed during the no-connect period. 


2) Regardless of quick restarts or transient network problems, sometimes a RIB 
dump and PEER UP is not needed.  It would be nice to pick up where we left off, 
if that is possible.  This should be per-peer instead of being binary at the 
session level due chatty peers causing an issue with buffering. This can 
include use-cases for logging, where the logging process does not actually care 
for a RIB dump at all.  Instead, it only wants new messages starting at BMP 
receiver connect time (based on peer and afi/safi).  

3) BMP receivers (like routers when needing to reapply a policy) sometimes need 
to get a refresh for a subset of peers.   For example, a DB change that results 
in some peers needing to be added again.   Currently, the method to refresh 
are: 

   a) go to the router and initiate a route-refresh, which is intrusive and
  requires auth to do this. Not great.
   b) reset the BMP TCP connection to trigger the router to refresh everything.
  This is not ideal as there can be hundreds of peers and only a small set 
needed to be refreshed. 


IMO, the same solution can be used to solve for all of the above.   I would 
like to see a new BMP message that the receiver sends on initial connect to 
indicate what's needed.  It's important to call out that not all peers (by 
afi/safi) are equal in terms of buffer exhaustion during connection loss to a 
receiver.   For example, link-state, public/private peering with customers, 
etc… do not have many updates over several minutes.  An all-or-nothing approach 
based on short-time is not a desired solution (IMO) and leaves many other RIB 
dump use-cases unaddressed.  

Thanks,
Tim


On 3/11/21, 8:45 PM, "GROW"  wrote:


Hi Jakob,
 
➢ When processes abort unexpectedly, loss must be assumed unless data integrity 
can be specifically proven.
 
Absolutely. We need to distinguish between application and transport. At 
transport we do have sequence numbers and integrity on transport is ensured. On 
BMP application it is not. Here we need to distinguish between BMP application 
and BMP session. In a previous message to you I wrote:
 
➢ What I 

Re: [GROW] is TCP the right layer for BMP session resumption?

[Thomas.Graf]

Hi Jakob,


  *   When processes abort unexpectedly, loss must be assumed unless data 
integrity can be specifically proven.

Absolutely. We need to distinguish between application and transport. At 
transport we do have sequence numbers and integrity on transport is ensured. On 
BMP application it is not. Here we need to distinguish between BMP application 
and BMP session. In a previous message to you I wrote:


  *   What I wondered if you could describe a bit more what benefit we would 
gain with BMP sequence numbers. At which point within the BMP client 
application loss technically could occur.

At BMP IETF hackathon where we BMP/BGP metric loss. As a tester I believe that 
first we need to describe the problem space carefully. Than analyze where, at 
which point, the sequence numbers should be applied. And then validate it with 
running code.
Best wishes
Thomas

From: Jakob Heitz (jheitz) 
Sent: Thursday, March 11, 2021 11:29 PM
To: Graf Thomas, INI-NET-TCZ-ZH1 ; 
rainsword.w...@huawei.com; rob...@raszuk.net; j...@dataplane.org
Cc: grow@ietf.org
Subject: RE: [GROW] is TCP the right layer for BMP session resumption?

With draft-tppy-bmp-seamless-session, if TCP session is re-established within 
the timeout value and buffer is not full, no message lost occurs
This is a leap of faith. How can you be sure that the receiver has not lost any 
messages, even if the TCP session ends in FIN?
When processes abort unexpectedly, loss must be assumed unless data integrity 
can be specifically proven.

Regards,
Jakob.

From: GROW mailto:grow-boun...@ietf.org>> On Behalf Of 
thomas.g...@swisscom.com
Sent: Thursday, March 11, 2021 4:59 AM
To: rainsword.w...@huawei.com; 
rob...@raszuk.net; 
j...@dataplane.org
Cc: grow@ietf.org
Subject: Re: [GROW] is TCP the right layer for BMP session resumption?

Hi Haibo,


  *   Now we want to keep the BMP session active even the TCP session is 
closed, I think it means the BMP session state separate from the TCP session.
For the BMP session closing it is delayed. Yes.


  *   And in this scenario, we don't know whether the last message is sent to 
the server OK or not.
No. Without draft-tppy-bmp-seamless-session all BMP messages after TCP 
transport session closes are lost. With draft-tppy-bmp-seamless-session, if TCP 
session is re-established within the timeout value and buffer is not full, no 
message lost occurs. If TCP session is re-established outside the timeout value 
or buffer is full, than BMP session is considered new and initial BMP 
route-monitoring initial RIB dump starts. Under any circumstances, BMP message 
lost should occur while BMP session is considered to be up. Even during 
re-establishment window.

Does that make sense now?

Best wishes
Thomas

From: Wanghaibo (Rainsword) 
mailto:rainsword.w...@huawei.com>>
Sent: Thursday, March 11, 2021 3:00 AM
To: Graf Thomas, INI-NET-TCZ-ZH1 
mailto:thomas.g...@swisscom.com>>; 
rob...@raszuk.net; 
j...@dataplane.org
Cc: grow@ietf.org
Subject: RE: [GROW] is TCP the right layer for BMP session resumption?

Hi Tomas,

 According to the RFC7854, the BMP session is closely bound to the TCP 
session. So the BMP session will  end when TCP is closed.
 Now we want to keep the BMP session active even the TCP session is 
closed, I think it means the BMP session state separate from the TCP session.
 And in this scenario, we don't know whether the last message is sent 
to the server OK or not.
If we don't accept this, we should use a mechanisms like sequence no. to ensure 
that. But it will cause the BMP more complex.

Regards,
Haibo

From: thomas.g...@swisscom.com 
[mailto:thomas.g...@swisscom.com]
Sent: Wednesday, March 10, 2021 11:11 PM
To: Wanghaibo (Rainsword) 
mailto:rainsword.w...@huawei.com>>; 
rob...@raszuk.net; 
j...@dataplane.org
Cc: grow@ietf.org
Subject: RE: [GROW] is TCP the right layer for BMP session resumption?

Hi Haibo,

Quite the contrary. draft-tppy-bmp-seamless-session is not about the separation 
of the transport session from the BMP session. It is about to delay the 
termination of the BMP session when transport session is closed and introducing 
a mechanism to re-establish the BMP session.

The authors chose a careful way not to re-invent the wheel. Use existing 
protocols, change as less as possible on the BMP application and preserve the 
original goal of BMP to be unidirectional. We believe by keeping the session 
handling on TCP transport, this goal can be best achieved. We are looking 
forward from the working group to receive feedback if they feel the same way or 
if the goal should be addressed rather on the application layer.

Best wishes
Thomas

From: 

Re: [GROW] is TCP the right layer for BMP session resumption?

[Jakob Heitz (jheitz)]

With draft-tppy-bmp-seamless-session, if TCP session is re-established within 
the timeout value and buffer is not full, no message lost occurs
This is a leap of faith. How can you be sure that the receiver has not lost any 
messages, even if the TCP session ends in FIN?
When processes abort unexpectedly, loss must be assumed unless data integrity 
can be specifically proven.

Regards,
Jakob.

From: GROW  On Behalf Of thomas.g...@swisscom.com
Sent: Thursday, March 11, 2021 4:59 AM
To: rainsword.w...@huawei.com; rob...@raszuk.net; j...@dataplane.org
Cc: grow@ietf.org
Subject: Re: [GROW] is TCP the right layer for BMP session resumption?

Hi Haibo,


  *   Now we want to keep the BMP session active even the TCP session is 
closed, I think it means the BMP session state separate from the TCP session.
For the BMP session closing it is delayed. Yes.


  *   And in this scenario, we don't know whether the last message is sent to 
the server OK or not.
No. Without draft-tppy-bmp-seamless-session all BMP messages after TCP 
transport session closes are lost. With draft-tppy-bmp-seamless-session, if TCP 
session is re-established within the timeout value and buffer is not full, no 
message lost occurs. If TCP session is re-established outside the timeout value 
or buffer is full, than BMP session is considered new and initial BMP 
route-monitoring initial RIB dump starts. Under any circumstances, BMP message 
lost should occur while BMP session is considered to be up. Even during 
re-establishment window.

Does that make sense now?

Best wishes
Thomas

From: Wanghaibo (Rainsword) 
mailto:rainsword.w...@huawei.com>>
Sent: Thursday, March 11, 2021 3:00 AM
To: Graf Thomas, INI-NET-TCZ-ZH1 
mailto:thomas.g...@swisscom.com>>; 
rob...@raszuk.net; 
j...@dataplane.org
Cc: grow@ietf.org
Subject: RE: [GROW] is TCP the right layer for BMP session resumption?

Hi Tomas,

 According to the RFC7854, the BMP session is closely bound to the TCP 
session. So the BMP session will  end when TCP is closed.
 Now we want to keep the BMP session active even the TCP session is 
closed, I think it means the BMP session state separate from the TCP session.
 And in this scenario, we don't know whether the last message is sent 
to the server OK or not.
If we don't accept this, we should use a mechanisms like sequence no. to ensure 
that. But it will cause the BMP more complex.

Regards,
Haibo

From: thomas.g...@swisscom.com 
[mailto:thomas.g...@swisscom.com]
Sent: Wednesday, March 10, 2021 11:11 PM
To: Wanghaibo (Rainsword) 
mailto:rainsword.w...@huawei.com>>; 
rob...@raszuk.net; 
j...@dataplane.org
Cc: grow@ietf.org
Subject: RE: [GROW] is TCP the right layer for BMP session resumption?

Hi Haibo,

Quite the contrary. draft-tppy-bmp-seamless-session is not about the separation 
of the transport session from the BMP session. It is about to delay the 
termination of the BMP session when transport session is closed and introducing 
a mechanism to re-establish the BMP session.

The authors chose a careful way not to re-invent the wheel. Use existing 
protocols, change as less as possible on the BMP application and preserve the 
original goal of BMP to be unidirectional. We believe by keeping the session 
handling on TCP transport, this goal can be best achieved. We are looking 
forward from the working group to receive feedback if they feel the same way or 
if the goal should be addressed rather on the application layer.

Best wishes
Thomas

From: Wanghaibo (Rainsword) 
mailto:rainsword.w...@huawei.com>>
Sent: Wednesday, March 10, 2021 12:48 PM
To: Graf Thomas, INI-NET-TCZ-ZH1 
mailto:thomas.g...@swisscom.com>>; 
rob...@raszuk.net; 
j...@dataplane.org
Cc: grow@ietf.org
Subject: RE: [GROW] is TCP the right layer for BMP session resumption?

Hi Tomas,

 I think the main problem is how to separate the BMP session with the 
transport session. Even we choose a stateless transport, we also need to use 
some mechanism to ensure the message is succeed send to the sever, e.g., use 
sequence number in BMP RM message.

Regards,
Haibo

From: GROW [mailto:grow-boun...@ietf.org] On Behalf Of 
thomas.g...@swisscom.com
Sent: Wednesday, March 10, 2021 2:21 PM
To: rob...@raszuk.net; 
j...@dataplane.org
Cc: grow@ietf.org
Subject: Re: [GROW] is TCP the right layer for BMP session resumption?

Hi John and Robert,

Speaking as a network operator. I absolutely agree on your thoughts that a 
stateless transport would be preferred over a stateful.

Best wishes
Thomas

From: GROW mailto:grow-boun...@ietf.org>> On Behalf Of 
Robert Raszuk
Sent: Tuesday, March 9, 2021 10:38 PM
To: 

Re: [GROW] is TCP the right layer for BMP session resumption?

[Wanghaibo (Rainsword)]

Hi Thomas,

   Yes,  it's clear.

Regards,
Haibo


--
王海波 Wang Haibo
Mobile: +86-13621091983
Email: rainsword.w...@huawei.com
发件人：Thomas.Graf 
收件人：Wanghaibo (Rainsword) ;robert 
;jtk 
抄 送：grow 
时 间：2021-03-11 21:00:05
主 题：RE: [GROW] is TCP the right layer for BMP session resumption?

Hi Haibo,


  *   Now we want to keep the BMP session active even the TCP session is 
closed, I think it means the BMP session state separate from the TCP session.
For the BMP session closing it is delayed. Yes.


  *   And in this scenario, we don’t know whether the last message is sent to 
the server OK or not.
No. Without draft-tppy-bmp-seamless-session all BMP messages after TCP 
transport session closes are lost. With draft-tppy-bmp-seamless-session, if TCP 
session is re-established within the timeout value and buffer is not full, no 
message lost occurs. If TCP session is re-established outside the timeout value 
or buffer is full, than BMP session is considered new and initial BMP 
route-monitoring initial RIB dump starts. Under any circumstances, BMP message 
lost should occur while BMP session is considered to be up. Even during 
re-establishment window.

Does that make sense now?

Best wishes
Thomas

From: Wanghaibo (Rainsword) 
Sent: Thursday, March 11, 2021 3:00 AM
To: Graf Thomas, INI-NET-TCZ-ZH1 ; rob...@raszuk.net; 
j...@dataplane.org
Cc: grow@ietf.org
Subject: RE: [GROW] is TCP the right layer for BMP session resumption?

Hi Tomas,

 According to the RFC7854, the BMP session is closely bound to the TCP 
session. So the BMP session will  end when TCP is closed.
 Now we want to keep the BMP session active even the TCP session is 
closed, I think it means the BMP session state separate from the TCP session.
 And in this scenario, we don’t know whether the last message is sent 
to the server OK or not.
If we don’t accept this, we should use a mechanisms like sequence no. to ensure 
that. But it will cause the BMP more complex.

Regards,
Haibo

From: thomas.g...@swisscom.com 
[mailto:thomas.g...@swisscom.com]
Sent: Wednesday, March 10, 2021 11:11 PM
To: Wanghaibo (Rainsword) 
mailto:rainsword.w...@huawei.com>>; 
rob...@raszuk.net; 
j...@dataplane.org
Cc: grow@ietf.org
Subject: RE: [GROW] is TCP the right layer for BMP session resumption?

Hi Haibo,

Quite the contrary. draft-tppy-bmp-seamless-session is not about the separation 
of the transport session from the BMP session. It is about to delay the 
termination of the BMP session when transport session is closed and introducing 
a mechanism to re-establish the BMP session.

The authors chose a careful way not to re-invent the wheel. Use existing 
protocols, change as less as possible on the BMP application and preserve the 
original goal of BMP to be unidirectional. We believe by keeping the session 
handling on TCP transport, this goal can be best achieved. We are looking 
forward from the working group to receive feedback if they feel the same way or 
if the goal should be addressed rather on the application layer.

Best wishes
Thomas

From: Wanghaibo (Rainsword) 
mailto:rainsword.w...@huawei.com>>
Sent: Wednesday, March 10, 2021 12:48 PM
To: Graf Thomas, INI-NET-TCZ-ZH1 
mailto:thomas.g...@swisscom.com>>; 
rob...@raszuk.net; 
j...@dataplane.org
Cc: grow@ietf.org
Subject: RE: [GROW] is TCP the right layer for BMP session resumption?

Hi Tomas,

 I think the main problem is how to separate the BMP session with the 
transport session. Even we choose a stateless transport, we also need to use 
some mechanism to ensure the message is succeed send to the sever, e.g., use 
sequence number in BMP RM message.

Regards,
Haibo

From: GROW [mailto:grow-boun...@ietf.org] On Behalf Of 
thomas.g...@swisscom.com
Sent: Wednesday, March 10, 2021 2:21 PM
To: rob...@raszuk.net; 
j...@dataplane.org
Cc: grow@ietf.org
Subject: Re: [GROW] is TCP the right layer for BMP session resumption?

Hi John and Robert,

Speaking as a network operator. I absolutely agree on your thoughts that a 
stateless transport would be preferred over a stateful.

Best wishes
Thomas

From: GROW mailto:grow-boun...@ietf.org>> On Behalf Of 
Robert Raszuk
Sent: Tuesday, March 9, 2021 10:38 PM
To: John Kristoff mailto:j...@dataplane.org>>
Cc: grow@ietf.org grow@ietf.org 
mailto:grow@ietf.org>>
Subject: Re: [GROW] is TCP the right layer for BMP session resumption?

I second John's comment with a bit more optimism.

As gRPC over QUIC is becoming a reality and de-facto messaging standard there 
is going to be hardly any choice for any router's vendor to resist to 

Re: [GROW] is TCP the right layer for BMP session resumption?

[Thomas.Graf]

Hi Haibo,


  *   Now we want to keep the BMP session active even the TCP session is 
closed, I think it means the BMP session state separate from the TCP session.
For the BMP session closing it is delayed. Yes.


  *   And in this scenario, we don't know whether the last message is sent to 
the server OK or not.
No. Without draft-tppy-bmp-seamless-session all BMP messages after TCP 
transport session closes are lost. With draft-tppy-bmp-seamless-session, if TCP 
session is re-established within the timeout value and buffer is not full, no 
message lost occurs. If TCP session is re-established outside the timeout value 
or buffer is full, than BMP session is considered new and initial BMP 
route-monitoring initial RIB dump starts. Under any circumstances, BMP message 
lost should occur while BMP session is considered to be up. Even during 
re-establishment window.

Does that make sense now?

Best wishes
Thomas

From: Wanghaibo (Rainsword) 
Sent: Thursday, March 11, 2021 3:00 AM
To: Graf Thomas, INI-NET-TCZ-ZH1 ; rob...@raszuk.net; 
j...@dataplane.org
Cc: grow@ietf.org
Subject: RE: [GROW] is TCP the right layer for BMP session resumption?

Hi Tomas,

 According to the RFC7854, the BMP session is closely bound to the TCP 
session. So the BMP session will  end when TCP is closed.
 Now we want to keep the BMP session active even the TCP session is 
closed, I think it means the BMP session state separate from the TCP session.
 And in this scenario, we don't know whether the last message is sent 
to the server OK or not.
If we don't accept this, we should use a mechanisms like sequence no. to ensure 
that. But it will cause the BMP more complex.

Regards,
Haibo

From: thomas.g...@swisscom.com 
[mailto:thomas.g...@swisscom.com]
Sent: Wednesday, March 10, 2021 11:11 PM
To: Wanghaibo (Rainsword) 
mailto:rainsword.w...@huawei.com>>; 
rob...@raszuk.net; 
j...@dataplane.org
Cc: grow@ietf.org
Subject: RE: [GROW] is TCP the right layer for BMP session resumption?

Hi Haibo,

Quite the contrary. draft-tppy-bmp-seamless-session is not about the separation 
of the transport session from the BMP session. It is about to delay the 
termination of the BMP session when transport session is closed and introducing 
a mechanism to re-establish the BMP session.

The authors chose a careful way not to re-invent the wheel. Use existing 
protocols, change as less as possible on the BMP application and preserve the 
original goal of BMP to be unidirectional. We believe by keeping the session 
handling on TCP transport, this goal can be best achieved. We are looking 
forward from the working group to receive feedback if they feel the same way or 
if the goal should be addressed rather on the application layer.

Best wishes
Thomas

From: Wanghaibo (Rainsword) 
mailto:rainsword.w...@huawei.com>>
Sent: Wednesday, March 10, 2021 12:48 PM
To: Graf Thomas, INI-NET-TCZ-ZH1 
mailto:thomas.g...@swisscom.com>>; 
rob...@raszuk.net; 
j...@dataplane.org
Cc: grow@ietf.org
Subject: RE: [GROW] is TCP the right layer for BMP session resumption?

Hi Tomas,

 I think the main problem is how to separate the BMP session with the 
transport session. Even we choose a stateless transport, we also need to use 
some mechanism to ensure the message is succeed send to the sever, e.g., use 
sequence number in BMP RM message.

Regards,
Haibo

From: GROW [mailto:grow-boun...@ietf.org] On Behalf Of 
thomas.g...@swisscom.com
Sent: Wednesday, March 10, 2021 2:21 PM
To: rob...@raszuk.net; 
j...@dataplane.org
Cc: grow@ietf.org
Subject: Re: [GROW] is TCP the right layer for BMP session resumption?

Hi John and Robert,

Speaking as a network operator. I absolutely agree on your thoughts that a 
stateless transport would be preferred over a stateful.

Best wishes
Thomas

From: GROW mailto:grow-boun...@ietf.org>> On Behalf Of 
Robert Raszuk
Sent: Tuesday, March 9, 2021 10:38 PM
To: John Kristoff mailto:j...@dataplane.org>>
Cc: grow@ietf.org grow@ietf.org 
mailto:grow@ietf.org>>
Subject: Re: [GROW] is TCP the right layer for BMP session resumption?

I second John's comment with a bit more optimism.

As gRPC over QUIC is becoming a reality and de-facto messaging standard there 
is going to be hardly any choice for any router's vendor to resist to implement 
it.

Best,
R.


On Tue, Mar 9, 2021 at 9:57 PM John Kristoff 
mailto:j...@dataplane.org>> wrote:
On Tue, 9 Mar 2021 20:44:18 +
"Jakob Heitz \(jheitz\)" 
mailto:40cisco@dmarc.ietf.org>> wrote:

> I've seen this session resumption technique in the '90s.
> BMP is a one-way protocol. The BMP server sends nothing.

I kind of wish my BMP router monitor was able 

Re: [GROW] is TCP the right layer for BMP session resumption?

[Thomas.Graf]

Hi Jakob,

All ack. Perfect. Thanks

Regards,
Thomas

-Original Message-
From: Jakob Heitz (jheitz)  
Sent: Thursday, March 11, 2021 3:17 AM
To: Graf Thomas, INI-NET-TCZ-ZH1 ; 
job=40fastly@dmarc.ietf.org
Cc: draft-tppy-bmp-seamless-sess...@ietf.org; grow@ietf.org
Subject: RE: [GROW] is TCP the right layer for BMP session resumption?

For the router, it's not about the time, its about the buffer space.
Configure the buffer space allowed to buffer incoming updates during the down 
time. If the buffer runs out, new sesssion.

Regards,
Jakob.

-Original Message-
From: thomas.g...@swisscom.com 
Sent: Wednesday, March 10, 2021 6:56 AM
To: Jakob Heitz (jheitz) ; job=40fastly@dmarc.ietf.org
Cc: draft-tppy-bmp-seamless-sess...@ietf.org; grow@ietf.org
Subject: RE: [GROW] is TCP the right layer for BMP session resumption?

Hi Jakob and Job,

Ack on all. I would define 60 seconds to be default and configurable.

Best wishes
Thomas

-Original Message-
From: Jakob Heitz (jheitz) 
Sent: Wednesday, March 10, 2021 1:12 PM
To: Job Snijders 
Cc: draft-tppy-bmp-seamless-sess...@ietf.org; grow@ietf.org
Subject: RE: [GROW] is TCP the right layer for BMP session resumption?

I would say 60 seconds or until the client runs out of configured buffer space 
to save messages that need to be sent to the session once the new session comes 
up.

Regards,
Jakob.

-Original Message-
From: Job Snijders 
Sent: Wednesday, March 10, 2021 1:04 AM
To: Jakob Heitz (jheitz) 
Cc: draft-tppy-bmp-seamless-sess...@ietf.org; grow@ietf.org
Subject: Re: [GROW] is TCP the right layer for BMP session resumption?

On Wed, Mar 10, 2021 at 03:08:34AM +, Jakob Heitz (jheitz) wrote:
> >From the BGP speaker (client) implementation point of view,
>
> I would do it like this:
> The client keeps a ring buffer of data it sent to the server.
> The bottom of the buffer is at a certain sequence number.
> As messages are created, that bottom moves up.
> If the server were to restart, it would send its last received 
> sequence number and session ID. If the buffer still contains the 
> sequence number, then you're in luck, else big bang restart.
> The content of the buffer could be optimized by retrieving some of the 
> information from the bgp table (adj-rib's are conceptual only) instead 
> of literally storing it. How and if any optimization is done is 
> implementation specific and not part of an RFC.

In the 'Sent: Tuesday, March 9, 2021 4:50 PM' email sequence/serial numbers are 
not required, only a 'session id' (which might remain the same over the 
lifetime of the BMP client's lifetime).

A full 'big bang' restart is achieved by the server disconnecting and allowing 
reconnection twice, within the 60 second window. When combined with 
TCP_FAST_OPEN resuming a session requires 2 packets (one each way) and 
restarting a session requires 4 packets.

This way BMP remains a 'read only' protocol, but I admit this is an 
unconventional approach.

Kind regards,

Job

smime.p7s
Description: S/MIME Cryptographic Signature
___
GROW mailing list
GROW@ietf.org
https://www.ietf.org/mailman/listinfo/grow

Re: [GROW] is TCP the right layer for BMP session resumption?

[Jakob Heitz (jheitz)]

For the router, it's not about the time, its about the buffer space.
Configure the buffer space allowed to buffer incoming updates during
the down time. If the buffer runs out, new sesssion.

Regards,
Jakob.

-Original Message-
From: thomas.g...@swisscom.com  
Sent: Wednesday, March 10, 2021 6:56 AM
To: Jakob Heitz (jheitz) ; job=40fastly@dmarc.ietf.org
Cc: draft-tppy-bmp-seamless-sess...@ietf.org; grow@ietf.org
Subject: RE: [GROW] is TCP the right layer for BMP session resumption?

Hi Jakob and Job,

Ack on all. I would define 60 seconds to be default and configurable.

Best wishes
Thomas

-Original Message-
From: Jakob Heitz (jheitz)  
Sent: Wednesday, March 10, 2021 1:12 PM
To: Job Snijders 
Cc: draft-tppy-bmp-seamless-sess...@ietf.org; grow@ietf.org
Subject: RE: [GROW] is TCP the right layer for BMP session resumption?

I would say 60 seconds or until the client runs out of configured buffer space 
to save messages that need to be sent to the session once the new session comes 
up.

Regards,
Jakob.

-Original Message-
From: Job Snijders 
Sent: Wednesday, March 10, 2021 1:04 AM
To: Jakob Heitz (jheitz) 
Cc: draft-tppy-bmp-seamless-sess...@ietf.org; grow@ietf.org
Subject: Re: [GROW] is TCP the right layer for BMP session resumption?

On Wed, Mar 10, 2021 at 03:08:34AM +, Jakob Heitz (jheitz) wrote:
> >From the BGP speaker (client) implementation point of view,
>
> I would do it like this:
> The client keeps a ring buffer of data it sent to the server.
> The bottom of the buffer is at a certain sequence number.
> As messages are created, that bottom moves up.
> If the server were to restart, it would send its last received 
> sequence number and session ID. If the buffer still contains the 
> sequence number, then you're in luck, else big bang restart.
> The content of the buffer could be optimized by retrieving some of the 
> information from the bgp table (adj-rib's are conceptual only) instead 
> of literally storing it. How and if any optimization is done is 
> implementation specific and not part of an RFC.

In the 'Sent: Tuesday, March 9, 2021 4:50 PM' email sequence/serial numbers are 
not required, only a 'session id' (which might remain the same over the 
lifetime of the BMP client's lifetime).

A full 'big bang' restart is achieved by the server disconnecting and allowing 
reconnection twice, within the 60 second window. When combined with 
TCP_FAST_OPEN resuming a session requires 2 packets (one each way) and 
restarting a session requires 4 packets.

This way BMP remains a 'read only' protocol, but I admit this is an 
unconventional approach.

Kind regards,

Job

___
GROW mailing list
GROW@ietf.org
https://www.ietf.org/mailman/listinfo/grow

Re: [GROW] is TCP the right layer for BMP session resumption?

[Wanghaibo (Rainsword)]

Hi Tomas,

 According to the RFC7854, the BMP session is closely bound to the TCP 
session. So the BMP session will  end when TCP is closed.
 Now we want to keep the BMP session active even the TCP session is 
closed, I think it means the BMP session state separate from the TCP session.
 And in this scenario, we don't know whether the last message is sent 
to the server OK or not.
If we don't accept this, we should use a mechanisms like sequence no. to ensure 
that. But it will cause the BMP more complex.

Regards,
Haibo

From: thomas.g...@swisscom.com [mailto:thomas.g...@swisscom.com]
Sent: Wednesday, March 10, 2021 11:11 PM
To: Wanghaibo (Rainsword) ; rob...@raszuk.net; 
j...@dataplane.org
Cc: grow@ietf.org
Subject: RE: [GROW] is TCP the right layer for BMP session resumption?

Hi Haibo,

Quite the contrary. draft-tppy-bmp-seamless-session is not about the separation 
of the transport session from the BMP session. It is about to delay the 
termination of the BMP session when transport session is closed and introducing 
a mechanism to re-establish the BMP session.

The authors chose a careful way not to re-invent the wheel. Use existing 
protocols, change as less as possible on the BMP application and preserve the 
original goal of BMP to be unidirectional. We believe by keeping the session 
handling on TCP transport, this goal can be best achieved. We are looking 
forward from the working group to receive feedback if they feel the same way or 
if the goal should be addressed rather on the application layer.

Best wishes
Thomas

From: Wanghaibo (Rainsword) 
mailto:rainsword.w...@huawei.com>>
Sent: Wednesday, March 10, 2021 12:48 PM
To: Graf Thomas, INI-NET-TCZ-ZH1 
mailto:thomas.g...@swisscom.com>>; 
rob...@raszuk.net; 
j...@dataplane.org
Cc: grow@ietf.org
Subject: RE: [GROW] is TCP the right layer for BMP session resumption?

Hi Tomas,

 I think the main problem is how to separate the BMP session with the 
transport session. Even we choose a stateless transport, we also need to use 
some mechanism to ensure the message is succeed send to the sever, e.g., use 
sequence number in BMP RM message.

Regards,
Haibo

From: GROW [mailto:grow-boun...@ietf.org] On Behalf Of 
thomas.g...@swisscom.com
Sent: Wednesday, March 10, 2021 2:21 PM
To: rob...@raszuk.net; 
j...@dataplane.org
Cc: grow@ietf.org
Subject: Re: [GROW] is TCP the right layer for BMP session resumption?

Hi John and Robert,

Speaking as a network operator. I absolutely agree on your thoughts that a 
stateless transport would be preferred over a stateful.

Best wishes
Thomas

From: GROW mailto:grow-boun...@ietf.org>> On Behalf Of 
Robert Raszuk
Sent: Tuesday, March 9, 2021 10:38 PM
To: John Kristoff mailto:j...@dataplane.org>>
Cc: grow@ietf.org grow@ietf.org 
mailto:grow@ietf.org>>
Subject: Re: [GROW] is TCP the right layer for BMP session resumption?

I second John's comment with a bit more optimism.

As gRPC over QUIC is becoming a reality and de-facto messaging standard there 
is going to be hardly any choice for any router's vendor to resist to implement 
it.

Best,
R.


On Tue, Mar 9, 2021 at 9:57 PM John Kristoff 
mailto:j...@dataplane.org>> wrote:
On Tue, 9 Mar 2021 20:44:18 +
"Jakob Heitz \(jheitz\)" 
mailto:40cisco@dmarc.ietf.org>> wrote:

> I've seen this session resumption technique in the '90s.
> BMP is a one-way protocol. The BMP server sends nothing.

I kind of wish my BMP router monitor was able to transport data over UDP
to the listening station like syslog and flow data.  I would have
especially liked this after that time a blocked TCP port and the
inability to opena TCP connection once caused my BMP monitor router
doing the active open to crash (known and now fixed bug).

> Thus adding this is a significant rework of BMP.

I assume my desire for UDP above will never happen as a result.  Oh
well.

John

___
GROW mailing list
GROW@ietf.org
https://www.ietf.org/mailman/listinfo/grow
___
GROW mailing list
GROW@ietf.org
https://www.ietf.org/mailman/listinfo/grow

Re: [GROW] is TCP the right layer for BMP session resumption?

[Thomas.Graf]

Hi Haibo,

Quite the contrary. draft-tppy-bmp-seamless-session is not about the separation 
of the transport session from the BMP session. It is about to delay the 
termination of the BMP session when transport session is closed and introducing 
a mechanism to re-establish the BMP session.

The authors chose a careful way not to re-invent the wheel. Use existing 
protocols, change as less as possible on the BMP application and preserve the 
original goal of BMP to be unidirectional. We believe by keeping the session 
handling on TCP transport, this goal can be best achieved. We are looking 
forward from the working group to receive feedback if they feel the same way or 
if the goal should be addressed rather on the application layer.

Best wishes
Thomas

From: Wanghaibo (Rainsword) 
Sent: Wednesday, March 10, 2021 12:48 PM
To: Graf Thomas, INI-NET-TCZ-ZH1 ; rob...@raszuk.net; 
j...@dataplane.org
Cc: grow@ietf.org
Subject: RE: [GROW] is TCP the right layer for BMP session resumption?

Hi Tomas,

 I think the main problem is how to separate the BMP session with the 
transport session. Even we choose a stateless transport, we also need to use 
some mechanism to ensure the message is succeed send to the sever, e.g., use 
sequence number in BMP RM message.

Regards,
Haibo

From: GROW [mailto:grow-boun...@ietf.org] On Behalf Of 
thomas.g...@swisscom.com
Sent: Wednesday, March 10, 2021 2:21 PM
To: rob...@raszuk.net; 
j...@dataplane.org
Cc: grow@ietf.org
Subject: Re: [GROW] is TCP the right layer for BMP session resumption?

Hi John and Robert,

Speaking as a network operator. I absolutely agree on your thoughts that a 
stateless transport would be preferred over a stateful.

Best wishes
Thomas

From: GROW mailto:grow-boun...@ietf.org>> On Behalf Of 
Robert Raszuk
Sent: Tuesday, March 9, 2021 10:38 PM
To: John Kristoff mailto:j...@dataplane.org>>
Cc: grow@ietf.org grow@ietf.org 
mailto:grow@ietf.org>>
Subject: Re: [GROW] is TCP the right layer for BMP session resumption?

I second John's comment with a bit more optimism.

As gRPC over QUIC is becoming a reality and de-facto messaging standard there 
is going to be hardly any choice for any router's vendor to resist to implement 
it.

Best,
R.


On Tue, Mar 9, 2021 at 9:57 PM John Kristoff 
mailto:j...@dataplane.org>> wrote:
On Tue, 9 Mar 2021 20:44:18 +
"Jakob Heitz \(jheitz\)" 
mailto:40cisco@dmarc.ietf.org>> wrote:

> I've seen this session resumption technique in the '90s.
> BMP is a one-way protocol. The BMP server sends nothing.

I kind of wish my BMP router monitor was able to transport data over UDP
to the listening station like syslog and flow data.  I would have
especially liked this after that time a blocked TCP port and the
inability to opena TCP connection once caused my BMP monitor router
doing the active open to crash (known and now fixed bug).

> Thus adding this is a significant rework of BMP.

I assume my desire for UDP above will never happen as a result.  Oh
well.

John

___
GROW mailing list
GROW@ietf.org
https://www.ietf.org/mailman/listinfo/grow


smime.p7s
Description: S/MIME Cryptographic Signature
___
GROW mailing list
GROW@ietf.org
https://www.ietf.org/mailman/listinfo/grow

Re: [GROW] is TCP the right layer for BMP session resumption?

[Thomas.Graf]

Hi Jakob and Job,

Ack on all. I would define 60 seconds to be default and configurable.

Best wishes
Thomas

-Original Message-
From: Jakob Heitz (jheitz)  
Sent: Wednesday, March 10, 2021 1:12 PM
To: Job Snijders 
Cc: draft-tppy-bmp-seamless-sess...@ietf.org; grow@ietf.org
Subject: RE: [GROW] is TCP the right layer for BMP session resumption?

I would say 60 seconds or until the client runs out of configured buffer space 
to save messages that need to be sent to the session once the new session comes 
up.

Regards,
Jakob.

-Original Message-
From: Job Snijders 
Sent: Wednesday, March 10, 2021 1:04 AM
To: Jakob Heitz (jheitz) 
Cc: draft-tppy-bmp-seamless-sess...@ietf.org; grow@ietf.org
Subject: Re: [GROW] is TCP the right layer for BMP session resumption?

On Wed, Mar 10, 2021 at 03:08:34AM +, Jakob Heitz (jheitz) wrote:
> >From the BGP speaker (client) implementation point of view,
>
> I would do it like this:
> The client keeps a ring buffer of data it sent to the server.
> The bottom of the buffer is at a certain sequence number.
> As messages are created, that bottom moves up.
> If the server were to restart, it would send its last received 
> sequence number and session ID. If the buffer still contains the 
> sequence number, then you're in luck, else big bang restart.
> The content of the buffer could be optimized by retrieving some of the 
> information from the bgp table (adj-rib's are conceptual only) instead 
> of literally storing it. How and if any optimization is done is 
> implementation specific and not part of an RFC.

In the 'Sent: Tuesday, March 9, 2021 4:50 PM' email sequence/serial numbers are 
not required, only a 'session id' (which might remain the same over the 
lifetime of the BMP client's lifetime).

A full 'big bang' restart is achieved by the server disconnecting and allowing 
reconnection twice, within the 60 second window. When combined with 
TCP_FAST_OPEN resuming a session requires 2 packets (one each way) and 
restarting a session requires 4 packets.

This way BMP remains a 'read only' protocol, but I admit this is an 
unconventional approach.

Kind regards,

Job

smime.p7s
Description: S/MIME Cryptographic Signature
___
GROW mailing list
GROW@ietf.org
https://www.ietf.org/mailman/listinfo/grow

Re: [GROW] is TCP the right layer for BMP session resumption?

[Thomas.Graf]

Hi Jakob,

Thats clear. Apology. I was not precise enough. I would prefer the reliability 
to be solved on application layer than on transport layer since in a large 
scale BMP data collection, multiple daemons collect the BMP messages and 
failover among can occur.

Best wishes
Thomas

From: Jakob Heitz (jheitz) 
Sent: Wednesday, March 10, 2021 7:47 AM
To: Graf Thomas, INI-NET-TCZ-ZH1 ; rob...@raszuk.net; 
j...@dataplane.org
Cc: grow@ietf.org
Subject: RE: [GROW] is TCP the right layer for BMP session resumption?

QUIC is not stateless.
BMP over QUIC is not BMP over UDP.
BMP requires reliable transfer.
The state to provide reliability must exist somewhere.
If not in TCP (or QUIC), then in the app.

Regards,
Jakob.

From: GROW mailto:grow-boun...@ietf.org>> On Behalf Of 
thomas.g...@swisscom.com
Sent: Tuesday, March 9, 2021 10:21 PM
To: rob...@raszuk.net; 
j...@dataplane.org
Cc: grow@ietf.org
Subject: Re: [GROW] is TCP the right layer for BMP session resumption?

Hi John and Robert,

Speaking as a network operator. I absolutely agree on your thoughts that a 
stateless transport would be preferred over a stateful.

Best wishes
Thomas

From: GROW mailto:grow-boun...@ietf.org>> On Behalf Of 
Robert Raszuk
Sent: Tuesday, March 9, 2021 10:38 PM
To: John Kristoff mailto:j...@dataplane.org>>
Cc: grow@ietf.org grow@ietf.org 
mailto:grow@ietf.org>>
Subject: Re: [GROW] is TCP the right layer for BMP session resumption?

I second John's comment with a bit more optimism.

As gRPC over QUIC is becoming a reality and de-facto messaging standard there 
is going to be hardly any choice for any router's vendor to resist to implement 
it.

Best,
R.


On Tue, Mar 9, 2021 at 9:57 PM John Kristoff 
mailto:j...@dataplane.org>> wrote:
On Tue, 9 Mar 2021 20:44:18 +
"Jakob Heitz \(jheitz\)" 
mailto:40cisco@dmarc.ietf.org>> wrote:

> I've seen this session resumption technique in the '90s.
> BMP is a one-way protocol. The BMP server sends nothing.

I kind of wish my BMP router monitor was able to transport data over UDP
to the listening station like syslog and flow data.  I would have
especially liked this after that time a blocked TCP port and the
inability to opena TCP connection once caused my BMP monitor router
doing the active open to crash (known and now fixed bug).

> Thus adding this is a significant rework of BMP.

I assume my desire for UDP above will never happen as a result.  Oh
well.

John

___
GROW mailing list
GROW@ietf.org
https://www.ietf.org/mailman/listinfo/grow


smime.p7s
Description: S/MIME Cryptographic Signature
___
GROW mailing list
GROW@ietf.org
https://www.ietf.org/mailman/listinfo/grow

Re: [GROW] is TCP the right layer for BMP session resumption?

[Thomas.Graf]

Hi Jakob,

Ack on all. The difference between sequence numbers in TCP transport and BMP 
application is clear. What I wondered if you could describe a bit more what 
benefit we would gain with BMP sequence numbers. At which point within the BMP 
client application loss technically could occur.

Best wishes
Thomas

-Original Message-
From: Jakob Heitz (jheitz)  
Sent: Wednesday, March 10, 2021 7:38 AM
To: Graf Thomas, INI-NET-TCZ-ZH1 ; 
job=40fastly@dmarc.ietf.org
Cc: draft-tppy-bmp-seamless-sess...@ietf.org; grow@ietf.org
Subject: RE: [GROW] is TCP the right layer for BMP session resumption?

TCP sequence numbers are for TCP only.
It would be nice if TCP were to acknowledge received data only after all 
application layers have fully processed it.
However, TCP sequence numbers are only for TCP.
The application cannot acknowledge full processing of received data back to TCP 
through the socket layer.
If the application layer wants to signal full processing of received data back 
to the sender, then it needs its own sequence numbers.

It's these sequence numbers that I want to be in 64 bits, not the session ID.

Storing the withdraw messages is an optimization that would work for monitor 
mode. In general, the sender has to store all data that has not been 
acknowledged at the application layer (not the TCP layer) if it is going to be 
retransmitted in any subsequent TCP session. In monitor mode, the sender can 
retrieve the non-withdraw messages from the information stored in the BGP table.

Regards,
Jakob.

-Original Message-
From: thomas.g...@swisscom.com 
Sent: Tuesday, March 9, 2021 10:19 PM
To: Jakob Heitz (jheitz) ; job=40fastly@dmarc.ietf.org
Cc: draft-tppy-bmp-seamless-sess...@ietf.org; grow@ietf.org
Subject: RE: [GROW] is TCP the right layer for BMP session resumption?

Hi Job and Jakob,

Many thanks for the good inputs which I consolidated in this reply.

In regards to TFO applicability to the BMP application.

During my initial research I was encouraged my section 6 of TFO RFC 7413 

https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftools.ietf.org%2Fhtml%2Frfc7413%23section-6data=04%7C01%7CThomas.Graf%40swisscom.com%7Cdba64e44152c438cbab708d8e38f7972%7C364e5b87c1c7420d9beec35d19b557a1%7C1%7C0%7C637509552641063005%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=fPQYYJPEYnfYwyC7bT4LZCfkUZUdlz1ZuLN9F3oT7Kg%3Dreserved=0

It is well understood that the original intend of the RFC is two fold:

- allow to re-establish a TCP session
- performance gain for short-lived connections

While the first is the motivation why TFO was chosen in this draft, the second 
is a welcomed by product where BMP could benefit from.

Regarding the TCP_FAST_OPEN cookie handling and the separation between 
application (BMP) and transport (TCP). The goal of the draft is that both 
sides, the BMP client and the BMP server can decide wherever the new transport 
session belongs to the previous BMP session or not. This is done on the BMP 
client side by either clearing the TCP_FAST_OPEN cookie for transport or not. 
The BMP client does not need to know the previous value. It needs only to 
distinguish between establish a new session or re-establish an existing 
session. Different to the BMP client, the BMP server does the same but instead 
of clearing the TCP_FAST_OPEN cookie for transport it sets a new, different to 
the previous, cookie.  

I like to take your input Job about the TFO applicability to the BMP 
application and do my due diligence by going to the TCPM working group and get 
their opinion. I will also do my own research on applications using TFO and for 
which purpose. I will present then that to the GROW list and the next GROW 
session. Does that make sense?

As you already pointed out the goal can be achieved not only on the TCP 
transport layer but also on the BMP application layer. There with the drawback 
that BMP is strictly speaking no longer unidirectional.

Here my proposal would be to extend the BMP Initiation Message with another TLV 
containing the BMP session identifier. I agree that the size should large 
enough to be unique and I take the input being 64 bit as a proposal. The client 
set's the BMP session identifier and the server stores it. When a BMP session 
is established, a new BMP session identifier is set be the client and stored at 
the BMP server. When the BMP client establishes the BMP session, the server 
decides wherever to reply with the previously stored value (signaling its 
state) or 0 (a new session). BMP client acts wherever on exporting its queue or 
start a new RIB dump depending if BMP session identifier is different to the 
previous or not.

Regarding the input from Jakob that not all the BMP message types should be 
queued, only BGP withdrawals. This is an interesting proposal I like to follow 
up and further like to understand it. If I understand correctly, only 
withdrawals would be 

Re: [GROW] is TCP the right layer for BMP session resumption?

[Jared Mauch]

A primary use-case of the BMP data is to provide information to a route 
collector/optimizer to determine what feasible paths may be sent to a router by 
these offline computational systems.  This requires a reliable transport where 
messages are delivered in order.

I understand others may be fine with missed or out of order messages, but this 
isn’t something that there is a lot of room to discuss.

I’m not a fan of QUIC but if it provides the ability to resume the session and 
preserve the order when there’s a [brief] network disruption that may be 
helpful for other use cases.

- Jared

> On Mar 10, 2021, at 1:47 AM, Jakob Heitz (jheitz) 
>  wrote:
> 
> QUIC is not stateless.
> BMP over QUIC is not BMP over UDP.
> BMP requires reliable transfer.
> The state to provide reliability must exist somewhere.
> If not in TCP (or QUIC), then in the app.
>  
> Regards,
> Jakob.
>  
> From: GROW  On Behalf Of thomas.g...@swisscom.com
> Sent: Tuesday, March 9, 2021 10:21 PM
> To: rob...@raszuk.net; j...@dataplane.org
> Cc: grow@ietf.org
> Subject: Re: [GROW] is TCP the right layer for BMP session resumption?
>  
> Hi John and Robert,
>  
> Speaking as a network operator. I absolutely agree on your thoughts that a 
> stateless transport would be preferred over a stateful.
>  
> Best wishes
> Thomas
>  
> From: GROW  On Behalf Of Robert Raszuk
> Sent: Tuesday, March 9, 2021 10:38 PM
> To: John Kristoff 
> Cc: grow@ietf.org grow@ietf.org 
> Subject: Re: [GROW] is TCP the right layer for BMP session resumption?
>  
> I second John's comment with a bit more optimism. 
>  
> As gRPC over QUIC is becoming a reality and de-facto messaging standard there 
> is going to be hardly any choice for any router's vendor to resist to 
> implement it. 
>  
> Best,
> R.
>  
>  
> On Tue, Mar 9, 2021 at 9:57 PM John Kristoff  wrote:
> On Tue, 9 Mar 2021 20:44:18 +
> "Jakob Heitz \(jheitz\)"  wrote:
> 
> > I've seen this session resumption technique in the '90s.
> > BMP is a one-way protocol. The BMP server sends nothing.
> 
> I kind of wish my BMP router monitor was able to transport data over UDP
> to the listening station like syslog and flow data.  I would have
> especially liked this after that time a blocked TCP port and the
> inability to opena TCP connection once caused my BMP monitor router
> doing the active open to crash (known and now fixed bug).
> 
> > Thus adding this is a significant rework of BMP.
> 
> I assume my desire for UDP above will never happen as a result.  Oh
> well.
> 
> John
> 
> ___
> GROW mailing list
> GROW@ietf.org
> https://www.ietf.org/mailman/listinfo/grow
> ___
> GROW mailing list
> GROW@ietf.org
> https://www.ietf.org/mailman/listinfo/grow

___
GROW mailing list
GROW@ietf.org
https://www.ietf.org/mailman/listinfo/grow

Re: [GROW] is TCP the right layer for BMP session resumption?

[Jakob Heitz (jheitz)]

I would say 60 seconds or until the client runs out of configured buffer space 
to save messages that need to be sent to the session once the new session comes 
up.

Regards,
Jakob.

-Original Message-
From: Job Snijders  
Sent: Wednesday, March 10, 2021 1:04 AM
To: Jakob Heitz (jheitz) 
Cc: draft-tppy-bmp-seamless-sess...@ietf.org; grow@ietf.org
Subject: Re: [GROW] is TCP the right layer for BMP session resumption?

On Wed, Mar 10, 2021 at 03:08:34AM +, Jakob Heitz (jheitz) wrote:
> >From the BGP speaker (client) implementation point of view,
>
> I would do it like this:
> The client keeps a ring buffer of data it sent to the server.
> The bottom of the buffer is at a certain sequence number.
> As messages are created, that bottom moves up.
> If the server were to restart, it would send its last
> received sequence number and session ID. If the buffer still
> contains the sequence number, then you're in luck, else
> big bang restart.
> The content of the buffer could be optimized by retrieving some
> of the information from the bgp table (adj-rib's are conceptual only)
> instead of literally storing it. How and if any optimization is
> done is implementation specific and not part of an RFC.

In the 'Sent: Tuesday, March 9, 2021 4:50 PM' email sequence/serial
numbers are not required, only a 'session id' (which might remain the
same over the lifetime of the BMP client's lifetime).

A full 'big bang' restart is achieved by the server disconnecting and
allowing reconnection twice, within the 60 second window. When combined
with TCP_FAST_OPEN resuming a session requires 2 packets (one each way)
and restarting a session requires 4 packets.

This way BMP remains a 'read only' protocol, but I admit this is
an unconventional approach.

Kind regards,

Job

___
GROW mailing list
GROW@ietf.org
https://www.ietf.org/mailman/listinfo/grow

Re: [GROW] is TCP the right layer for BMP session resumption?

[Wanghaibo (Rainsword)]

Hi Tomas,

 I think the main problem is how to separate the BMP session with the 
transport session. Even we choose a stateless transport, we also need to use 
some mechanism to ensure the message is succeed send to the sever, e.g., use 
sequence number in BMP RM message.

Regards,
Haibo

From: GROW [mailto:grow-boun...@ietf.org] On Behalf Of thomas.g...@swisscom.com
Sent: Wednesday, March 10, 2021 2:21 PM
To: rob...@raszuk.net; j...@dataplane.org
Cc: grow@ietf.org
Subject: Re: [GROW] is TCP the right layer for BMP session resumption?

Hi John and Robert,

Speaking as a network operator. I absolutely agree on your thoughts that a 
stateless transport would be preferred over a stateful.

Best wishes
Thomas

From: GROW mailto:grow-boun...@ietf.org>> On Behalf Of 
Robert Raszuk
Sent: Tuesday, March 9, 2021 10:38 PM
To: John Kristoff mailto:j...@dataplane.org>>
Cc: grow@ietf.org grow@ietf.org 
mailto:grow@ietf.org>>
Subject: Re: [GROW] is TCP the right layer for BMP session resumption?

I second John's comment with a bit more optimism.

As gRPC over QUIC is becoming a reality and de-facto messaging standard there 
is going to be hardly any choice for any router's vendor to resist to implement 
it.

Best,
R.


On Tue, Mar 9, 2021 at 9:57 PM John Kristoff 
mailto:j...@dataplane.org>> wrote:
On Tue, 9 Mar 2021 20:44:18 +
"Jakob Heitz \(jheitz\)" 
mailto:40cisco@dmarc.ietf.org>> wrote:

> I've seen this session resumption technique in the '90s.
> BMP is a one-way protocol. The BMP server sends nothing.

I kind of wish my BMP router monitor was able to transport data over UDP
to the listening station like syslog and flow data.  I would have
especially liked this after that time a blocked TCP port and the
inability to opena TCP connection once caused my BMP monitor router
doing the active open to crash (known and now fixed bug).

> Thus adding this is a significant rework of BMP.

I assume my desire for UDP above will never happen as a result.  Oh
well.

John

___
GROW mailing list
GROW@ietf.org
https://www.ietf.org/mailman/listinfo/grow
___
GROW mailing list
GROW@ietf.org
https://www.ietf.org/mailman/listinfo/grow

Re: [GROW] is TCP the right layer for BMP session resumption?

[Douglas Fischer]

sed ´s/sensible/sensitive´

In pt_BR both words are "Sensível", and what defines the real meaning is
the phrase context.
Same with Safety and Security, both are "segurança" in pt_BR.

Dã... Sorry!

Em qua., 10 de mar. de 2021 às 07:16, Douglas Fischer <
fischerdoug...@gmail.com> escreveu:

> I'm not sure about what I'm going to say... But.
>
> BMP would transfer sensible data.
> Then some cryptographic layer would be recommended/necessary.
>
> Considering that, following on this approach of "fast connection", will
> not have time/space to negotiate some crypto on those fast reconnections.
> So I presume something above(separate from the transport layer) will need
> to deal with that cryptographic layer, right?
>
>
>
>
> Em qua., 10 de mar. de 2021 às 06:04, Job Snijders  40fastly@dmarc.ietf.org> escreveu:
>
>> On Wed, Mar 10, 2021 at 03:08:34AM +, Jakob Heitz (jheitz) wrote:
>> > >From the BGP speaker (client) implementation point of view,
>> >
>> > I would do it like this:
>> > The client keeps a ring buffer of data it sent to the server.
>> > The bottom of the buffer is at a certain sequence number.
>> > As messages are created, that bottom moves up.
>> > If the server were to restart, it would send its last
>> > received sequence number and session ID. If the buffer still
>> > contains the sequence number, then you're in luck, else
>> > big bang restart.
>> > The content of the buffer could be optimized by retrieving some
>> > of the information from the bgp table (adj-rib's are conceptual only)
>> > instead of literally storing it. How and if any optimization is
>> > done is implementation specific and not part of an RFC.
>>
>> In the 'Sent: Tuesday, March 9, 2021 4:50 PM' email sequence/serial
>> numbers are not required, only a 'session id' (which might remain the
>> same over the lifetime of the BMP client's lifetime).
>>
>> A full 'big bang' restart is achieved by the server disconnecting and
>> allowing reconnection twice, within the 60 second window. When combined
>> with TCP_FAST_OPEN resuming a session requires 2 packets (one each way)
>> and restarting a session requires 4 packets.
>>
>> This way BMP remains a 'read only' protocol, but I admit this is
>> an unconventional approach.
>>
>> Kind regards,
>>
>> Job
>>
>> ___
>> GROW mailing list
>> GROW@ietf.org
>> https://www.ietf.org/mailman/listinfo/grow
>>
>
>
> --
> Douglas Fernando Fischer
> Engº de Controle e Automação
>


-- 
Douglas Fernando Fischer
Engº de Controle e Automação
___
GROW mailing list
GROW@ietf.org
https://www.ietf.org/mailman/listinfo/grow

Re: [GROW] is TCP the right layer for BMP session resumption?

[Douglas Fischer]

I'm not sure about what I'm going to say... But.

BMP would transfer sensible data.
Then some cryptographic layer would be recommended/necessary.

Considering that, following on this approach of "fast connection", will not
have time/space to negotiate some crypto on those fast reconnections.
So I presume something above(separate from the transport layer) will need
to deal with that cryptographic layer, right?




Em qua., 10 de mar. de 2021 às 06:04, Job Snijders  escreveu:

> On Wed, Mar 10, 2021 at 03:08:34AM +, Jakob Heitz (jheitz) wrote:
> > >From the BGP speaker (client) implementation point of view,
> >
> > I would do it like this:
> > The client keeps a ring buffer of data it sent to the server.
> > The bottom of the buffer is at a certain sequence number.
> > As messages are created, that bottom moves up.
> > If the server were to restart, it would send its last
> > received sequence number and session ID. If the buffer still
> > contains the sequence number, then you're in luck, else
> > big bang restart.
> > The content of the buffer could be optimized by retrieving some
> > of the information from the bgp table (adj-rib's are conceptual only)
> > instead of literally storing it. How and if any optimization is
> > done is implementation specific and not part of an RFC.
>
> In the 'Sent: Tuesday, March 9, 2021 4:50 PM' email sequence/serial
> numbers are not required, only a 'session id' (which might remain the
> same over the lifetime of the BMP client's lifetime).
>
> A full 'big bang' restart is achieved by the server disconnecting and
> allowing reconnection twice, within the 60 second window. When combined
> with TCP_FAST_OPEN resuming a session requires 2 packets (one each way)
> and restarting a session requires 4 packets.
>
> This way BMP remains a 'read only' protocol, but I admit this is
> an unconventional approach.
>
> Kind regards,
>
> Job
>
> ___
> GROW mailing list
> GROW@ietf.org
> https://www.ietf.org/mailman/listinfo/grow
>


-- 
Douglas Fernando Fischer
Engº de Controle e Automação
___
GROW mailing list
GROW@ietf.org
https://www.ietf.org/mailman/listinfo/grow

Re: [GROW] is TCP the right layer for BMP session resumption?

[Job Snijders]

On Wed, Mar 10, 2021 at 03:08:34AM +, Jakob Heitz (jheitz) wrote:
> >From the BGP speaker (client) implementation point of view,
>
> I would do it like this:
> The client keeps a ring buffer of data it sent to the server.
> The bottom of the buffer is at a certain sequence number.
> As messages are created, that bottom moves up.
> If the server were to restart, it would send its last
> received sequence number and session ID. If the buffer still
> contains the sequence number, then you're in luck, else
> big bang restart.
> The content of the buffer could be optimized by retrieving some
> of the information from the bgp table (adj-rib's are conceptual only)
> instead of literally storing it. How and if any optimization is
> done is implementation specific and not part of an RFC.

In the 'Sent: Tuesday, March 9, 2021 4:50 PM' email sequence/serial
numbers are not required, only a 'session id' (which might remain the
same over the lifetime of the BMP client's lifetime).

A full 'big bang' restart is achieved by the server disconnecting and
allowing reconnection twice, within the 60 second window. When combined
with TCP_FAST_OPEN resuming a session requires 2 packets (one each way)
and restarting a session requires 4 packets.

This way BMP remains a 'read only' protocol, but I admit this is
an unconventional approach.

Kind regards,

Job

___
GROW mailing list
GROW@ietf.org
https://www.ietf.org/mailman/listinfo/grow

Re: [GROW] is TCP the right layer for BMP session resumption?

[Jakob Heitz (jheitz)]

QUIC is not stateless.
BMP over QUIC is not BMP over UDP.
BMP requires reliable transfer.
The state to provide reliability must exist somewhere.
If not in TCP (or QUIC), then in the app.

Regards,
Jakob.

From: GROW  On Behalf Of thomas.g...@swisscom.com
Sent: Tuesday, March 9, 2021 10:21 PM
To: rob...@raszuk.net; j...@dataplane.org
Cc: grow@ietf.org
Subject: Re: [GROW] is TCP the right layer for BMP session resumption?

Hi John and Robert,

Speaking as a network operator. I absolutely agree on your thoughts that a 
stateless transport would be preferred over a stateful.

Best wishes
Thomas

From: GROW mailto:grow-boun...@ietf.org>> On Behalf Of 
Robert Raszuk
Sent: Tuesday, March 9, 2021 10:38 PM
To: John Kristoff mailto:j...@dataplane.org>>
Cc: grow@ietf.org grow@ietf.org 
mailto:grow@ietf.org>>
Subject: Re: [GROW] is TCP the right layer for BMP session resumption?

I second John's comment with a bit more optimism.

As gRPC over QUIC is becoming a reality and de-facto messaging standard there 
is going to be hardly any choice for any router's vendor to resist to implement 
it.

Best,
R.


On Tue, Mar 9, 2021 at 9:57 PM John Kristoff 
mailto:j...@dataplane.org>> wrote:
On Tue, 9 Mar 2021 20:44:18 +
"Jakob Heitz \(jheitz\)" 
mailto:40cisco@dmarc.ietf.org>> wrote:

> I've seen this session resumption technique in the '90s.
> BMP is a one-way protocol. The BMP server sends nothing.

I kind of wish my BMP router monitor was able to transport data over UDP
to the listening station like syslog and flow data.  I would have
especially liked this after that time a blocked TCP port and the
inability to opena TCP connection once caused my BMP monitor router
doing the active open to crash (known and now fixed bug).

> Thus adding this is a significant rework of BMP.

I assume my desire for UDP above will never happen as a result.  Oh
well.

John

___
GROW mailing list
GROW@ietf.org
https://www.ietf.org/mailman/listinfo/grow
___
GROW mailing list
GROW@ietf.org
https://www.ietf.org/mailman/listinfo/grow

Re: [GROW] is TCP the right layer for BMP session resumption?

[Jakob Heitz (jheitz)]

TCP sequence numbers are for TCP only.
It would be nice if TCP were to acknowledge received data only after 
all application layers have fully processed it.
However, TCP sequence numbers are only for TCP.
The application cannot acknowledge full processing of received data
back to TCP through the socket layer.
If the application layer wants to signal full processing of received
data back to the sender, then it needs its own sequence numbers.

It's these sequence numbers that I want to be in 64 bits,
not the session ID.

Storing the withdraw messages is an optimization that would work
for monitor mode. In general, the sender has to store all data
that has not been acknowledged at the application layer
(not the TCP layer) if it is going to be retransmitted in any
subsequent TCP session. In monitor mode, the sender can retrieve
the non-withdraw messages from the information stored in the BGP table.

Regards,
Jakob.

-Original Message-
From: thomas.g...@swisscom.com  
Sent: Tuesday, March 9, 2021 10:19 PM
To: Jakob Heitz (jheitz) ; job=40fastly@dmarc.ietf.org
Cc: draft-tppy-bmp-seamless-sess...@ietf.org; grow@ietf.org
Subject: RE: [GROW] is TCP the right layer for BMP session resumption?

Hi Job and Jakob,

Many thanks for the good inputs which I consolidated in this reply.

In regards to TFO applicability to the BMP application.

During my initial research I was encouraged my section 6 of TFO RFC 7413 

https://tools.ietf.org/html/rfc7413#section-6

It is well understood that the original intend of the RFC is two fold:

- allow to re-establish a TCP session
- performance gain for short-lived connections

While the first is the motivation why TFO was chosen in this draft, the second 
is a welcomed by product where BMP could benefit from.

Regarding the TCP_FAST_OPEN cookie handling and the separation between 
application (BMP) and transport (TCP). The goal of the draft is that both 
sides, the BMP client and the BMP server can decide wherever the new transport 
session belongs to the previous BMP session or not. This is done on the BMP 
client side by either clearing the TCP_FAST_OPEN cookie for transport or not. 
The BMP client does not need to know the previous value. It needs only to 
distinguish between establish a new session or re-establish an existing 
session. Different to the BMP client, the BMP server does the same but instead 
of clearing the TCP_FAST_OPEN cookie for transport it sets a new, different to 
the previous, cookie.  

I like to take your input Job about the TFO applicability to the BMP 
application and do my due diligence by going to the TCPM working group and get 
their opinion. I will also do my own research on applications using TFO and for 
which purpose. I will present then that to the GROW list and the next GROW 
session. Does that make sense?

As you already pointed out the goal can be achieved not only on the TCP 
transport layer but also on the BMP application layer. There with the drawback 
that BMP is strictly speaking no longer unidirectional.

Here my proposal would be to extend the BMP Initiation Message with another TLV 
containing the BMP session identifier. I agree that the size should large 
enough to be unique and I take the input being 64 bit as a proposal. The client 
set's the BMP session identifier and the server stores it. When a BMP session 
is established, a new BMP session identifier is set be the client and stored at 
the BMP server. When the BMP client establishes the BMP session, the server 
decides wherever to reply with the previously stored value (signaling its 
state) or 0 (a new session). BMP client acts wherever on exporting its queue or 
start a new RIB dump depending if BMP session identifier is different to the 
previous or not.

Regarding the input from Jakob that not all the BMP message types should be 
queued, only BGP withdrawals. This is an interesting proposal I like to follow 
up and further like to understand it. If I understand correctly, only 
withdrawals would be queued during the re-establish time window and updates 
would be locally generated for the re-establish time window once the BMP 
session is re-established. Correct? 

Regarding the proposal of sequence numbers. It would imply that the BMP Common 
Header needs to be extended. Here I like to hear your thoughts why a session 
identifier is not enough and what benefit a sequence number would bring on the 
application layer when we already have sequence numbers on the TCP transport 
session. As you might recall, one of the objectives of the BMP hackathon was to 
detect loss of BMP messages.

Also many thanks to Jeff about the feedback in regards to BGP NSR. I agree that 
solving the goal on a TCP transport layer would prevent implications onto 
BGP/BMP application in such condition when BGP process is being restarted.

Best wishes
Thomas

-Original Message-
From: Jakob Heitz (jheitz)  
Sent: Wednesday, March 10, 2021 4:09 AM
To: Job Snijders 
Cc: 

Re: [GROW] is TCP the right layer for BMP session resumption?

[Thomas.Graf]

Hi John and Robert,

Speaking as a network operator. I absolutely agree on your thoughts that a 
stateless transport would be preferred over a stateful.

Best wishes
Thomas

From: GROW  On Behalf Of Robert Raszuk
Sent: Tuesday, March 9, 2021 10:38 PM
To: John Kristoff 
Cc: grow@ietf.org grow@ietf.org 
Subject: Re: [GROW] is TCP the right layer for BMP session resumption?

I second John's comment with a bit more optimism.

As gRPC over QUIC is becoming a reality and de-facto messaging standard there 
is going to be hardly any choice for any router's vendor to resist to implement 
it.

Best,
R.


On Tue, Mar 9, 2021 at 9:57 PM John Kristoff 
mailto:j...@dataplane.org>> wrote:
On Tue, 9 Mar 2021 20:44:18 +
"Jakob Heitz \(jheitz\)" 
mailto:40cisco@dmarc.ietf.org>> wrote:

> I've seen this session resumption technique in the '90s.
> BMP is a one-way protocol. The BMP server sends nothing.

I kind of wish my BMP router monitor was able to transport data over UDP
to the listening station like syslog and flow data.  I would have
especially liked this after that time a blocked TCP port and the
inability to opena TCP connection once caused my BMP monitor router
doing the active open to crash (known and now fixed bug).

> Thus adding this is a significant rework of BMP.

I assume my desire for UDP above will never happen as a result.  Oh
well.

John

___
GROW mailing list
GROW@ietf.org
https://www.ietf.org/mailman/listinfo/grow


smime.p7s
Description: S/MIME Cryptographic Signature
___
GROW mailing list
GROW@ietf.org
https://www.ietf.org/mailman/listinfo/grow

Re: [GROW] is TCP the right layer for BMP session resumption?

[Thomas.Graf]

Hi Job and Jakob,

Many thanks for the good inputs which I consolidated in this reply.

In regards to TFO applicability to the BMP application.

During my initial research I was encouraged my section 6 of TFO RFC 7413 

https://tools.ietf.org/html/rfc7413#section-6

It is well understood that the original intend of the RFC is two fold:

- allow to re-establish a TCP session
- performance gain for short-lived connections

While the first is the motivation why TFO was chosen in this draft, the second 
is a welcomed by product where BMP could benefit from.

Regarding the TCP_FAST_OPEN cookie handling and the separation between 
application (BMP) and transport (TCP). The goal of the draft is that both 
sides, the BMP client and the BMP server can decide wherever the new transport 
session belongs to the previous BMP session or not. This is done on the BMP 
client side by either clearing the TCP_FAST_OPEN cookie for transport or not. 
The BMP client does not need to know the previous value. It needs only to 
distinguish between establish a new session or re-establish an existing 
session. Different to the BMP client, the BMP server does the same but instead 
of clearing the TCP_FAST_OPEN cookie for transport it sets a new, different to 
the previous, cookie.  

I like to take your input Job about the TFO applicability to the BMP 
application and do my due diligence by going to the TCPM working group and get 
their opinion. I will also do my own research on applications using TFO and for 
which purpose. I will present then that to the GROW list and the next GROW 
session. Does that make sense?

As you already pointed out the goal can be achieved not only on the TCP 
transport layer but also on the BMP application layer. There with the drawback 
that BMP is strictly speaking no longer unidirectional.

Here my proposal would be to extend the BMP Initiation Message with another TLV 
containing the BMP session identifier. I agree that the size should large 
enough to be unique and I take the input being 64 bit as a proposal. The client 
set's the BMP session identifier and the server stores it. When a BMP session 
is established, a new BMP session identifier is set be the client and stored at 
the BMP server. When the BMP client establishes the BMP session, the server 
decides wherever to reply with the previously stored value (signaling its 
state) or 0 (a new session). BMP client acts wherever on exporting its queue or 
start a new RIB dump depending if BMP session identifier is different to the 
previous or not.

Regarding the input from Jakob that not all the BMP message types should be 
queued, only BGP withdrawals. This is an interesting proposal I like to follow 
up and further like to understand it. If I understand correctly, only 
withdrawals would be queued during the re-establish time window and updates 
would be locally generated for the re-establish time window once the BMP 
session is re-established. Correct? 

Regarding the proposal of sequence numbers. It would imply that the BMP Common 
Header needs to be extended. Here I like to hear your thoughts why a session 
identifier is not enough and what benefit a sequence number would bring on the 
application layer when we already have sequence numbers on the TCP transport 
session. As you might recall, one of the objectives of the BMP hackathon was to 
detect loss of BMP messages.

Also many thanks to Jeff about the feedback in regards to BGP NSR. I agree that 
solving the goal on a TCP transport layer would prevent implications onto 
BGP/BMP application in such condition when BGP process is being restarted.

Best wishes
Thomas

-Original Message-
From: Jakob Heitz (jheitz)  
Sent: Wednesday, March 10, 2021 4:09 AM
To: Job Snijders 
Cc: draft-tppy-bmp-seamless-sess...@ietf.org; grow@ietf.org
Subject: RE: [GROW] is TCP the right layer for BMP session resumption?

>From the BGP speaker (client) implementation point of view,
I would do it like this:
The client keeps a ring buffer of data it sent to the server.
The bottom of the buffer is at a certain sequence number.
As messages are created, that bottom moves up.
If the server were to restart, it would send its last received sequence number 
and session ID. If the buffer still contains the sequence number, then you're 
in luck, else big bang restart.
The content of the buffer could be optimized by retrieving some of the 
information from the bgp table (adj-rib's are conceptual only) instead of 
literally storing it. How and if any optimization is done is implementation 
specific and not part of an RFC.

Regards,
Jakob.

-Original Message-
From: Job Snijders 
Sent: Tuesday, March 9, 2021 4:50 PM
To: Jakob Heitz (jheitz) 
Cc: draft-tppy-bmp-seamless-sess...@ietf.org; grow@ietf.org
Subject: Re: [GROW] is TCP the right layer for BMP session resumption?

On Tue, Mar 09, 2021 at 08:44:18PM +, Jakob Heitz (jheitz) wrote:
> BMP is a one-way protocol. The BMP server sends nothing.

In the 

Re: [GROW] is TCP the right layer for BMP session resumption?

[Jakob Heitz (jheitz)]

>From the BGP speaker (client) implementation point of view,
I would do it like this:
The client keeps a ring buffer of data it sent to the server.
The bottom of the buffer is at a certain sequence number.
As messages are created, that bottom moves up.
If the server were to restart, it would send its last
received sequence number and session ID. If the buffer still
contains the sequence number, then you're in luck, else
big bang restart.
The content of the buffer could be optimized by retrieving some
of the information from the bgp table (adj-rib's are conceptual only)
instead of literally storing it. How and if any optimization is
done is implementation specific and not part of an RFC.

Regards,
Jakob.

-Original Message-
From: Job Snijders  
Sent: Tuesday, March 9, 2021 4:50 PM
To: Jakob Heitz (jheitz) 
Cc: draft-tppy-bmp-seamless-sess...@ietf.org; grow@ietf.org
Subject: Re: [GROW] is TCP the right layer for BMP session resumption?

On Tue, Mar 09, 2021 at 08:44:18PM +, Jakob Heitz (jheitz) wrote:
> BMP is a one-way protocol. The BMP server sends nothing.

In the proposal at hand, the BMP server would send a client-specific
TCP_FAST_OPEN cookie (on top of TCP ACKs), and possibly eventually a TCP
RST, which is slightly more than 'nothing'... :-)

As TCP_FAST_OPEN already is a published RFC, existing BMP clients &
servers are free and able to opportunistically use TCP Fast Open. For
the sake of conversation I'll consider TCP_FAST_OPEN out-of-scope for
BMP and GROW in the rest this email.

BMP - one way protocol on reliable transport: are successive RSTs a signal?
===

In a one-way protocol where the recipient essentially is limited to 'TCP
ACK' and 'TCP RST' as response options, would it perhaps make sense to
outline a BMP protocol where both BMP client and BMP server assume more
delibrate intent from the timing of TCP reconnection events?

If the BMP client includes a 'session_id' message as its first message
towards the BMP server, then when the client loses its connection to the
BMP server, it can continue buffering messages destined for that
specific BMP server linked to the previously sent 'session_id'.

Then, if the BMP client manages to reconnect to the BMP server within 60
seconds, the client will flush all buffered messages associated with the
session_id also communicated in prior BMP sessions.

However if the BMP server closes the TCP session within that 60 seconds
buffer window, a subsequent successful reconnection would result
in the BGP client sending a new session_id followed by all 'Peer Up'
messages, because the previous BMP session (and buffer) were
terminated.

The BMP server can immediately disconnect when it receives a BMP
session_id it does not recognize (by issuing a TCP RST). When a BMP
client receives the 'second' TCP RST within 60 seconds, it can choose to
reconnect following an linear backoff model and for the duration of wait
periods exceeding 1 minute not bother buffering for 'unreachable' BMP
servers.

The heuristic is that the BMP client considers the BMP session
'resumed', iff a BMP server doesn't disconnect within 60 seconds.

The BMP server can use the session_id as input to its decision process
whether to disconnect within 60 seconds or not.

Blink once the BMP session survives... blink twice, game over!

Kind regards,

Job

___
GROW mailing list
GROW@ietf.org
https://www.ietf.org/mailman/listinfo/grow

Re: [GROW] is TCP the right layer for BMP session resumption?

[Job Snijders]

On Tue, Mar 09, 2021 at 08:44:18PM +, Jakob Heitz (jheitz) wrote:
> BMP is a one-way protocol. The BMP server sends nothing.

In the proposal at hand, the BMP server would send a client-specific
TCP_FAST_OPEN cookie (on top of TCP ACKs), and possibly eventually a TCP
RST, which is slightly more than 'nothing'... :-)

As TCP_FAST_OPEN already is a published RFC, existing BMP clients &
servers are free and able to opportunistically use TCP Fast Open. For
the sake of conversation I'll consider TCP_FAST_OPEN out-of-scope for
BMP and GROW in the rest this email.

BMP - one way protocol on reliable transport: are successive RSTs a signal?
===

In a one-way protocol where the recipient essentially is limited to 'TCP
ACK' and 'TCP RST' as response options, would it perhaps make sense to
outline a BMP protocol where both BMP client and BMP server assume more
delibrate intent from the timing of TCP reconnection events?

If the BMP client includes a 'session_id' message as its first message
towards the BMP server, then when the client loses its connection to the
BMP server, it can continue buffering messages destined for that
specific BMP server linked to the previously sent 'session_id'.

Then, if the BMP client manages to reconnect to the BMP server within 60
seconds, the client will flush all buffered messages associated with the
session_id also communicated in prior BMP sessions.

However if the BMP server closes the TCP session within that 60 seconds
buffer window, a subsequent successful reconnection would result
in the BGP client sending a new session_id followed by all 'Peer Up'
messages, because the previous BMP session (and buffer) were
terminated.

The BMP server can immediately disconnect when it receives a BMP
session_id it does not recognize (by issuing a TCP RST). When a BMP
client receives the 'second' TCP RST within 60 seconds, it can choose to
reconnect following an linear backoff model and for the duration of wait
periods exceeding 1 minute not bother buffering for 'unreachable' BMP
servers.

The heuristic is that the BMP client considers the BMP session
'resumed', iff a BMP server doesn't disconnect within 60 seconds.

The BMP server can use the session_id as input to its decision process
whether to disconnect within 60 seconds or not.

Blink once the BMP session survives... blink twice, game over!

Kind regards,

Job

___
GROW mailing list
GROW@ietf.org
https://www.ietf.org/mailman/listinfo/grow

Re: [GROW] is TCP the right layer for BMP session resumption?

[Robert Raszuk]

I second John's comment with a bit more optimism.

As gRPC over QUIC is becoming a reality and de-facto messaging standard
there is going to be hardly any choice for any router's vendor to resist to
implement it.

Best,
R.


On Tue, Mar 9, 2021 at 9:57 PM John Kristoff  wrote:

> On Tue, 9 Mar 2021 20:44:18 +
> "Jakob Heitz \(jheitz\)"  wrote:
>
> > I've seen this session resumption technique in the '90s.
> > BMP is a one-way protocol. The BMP server sends nothing.
>
> I kind of wish my BMP router monitor was able to transport data over UDP
> to the listening station like syslog and flow data.  I would have
> especially liked this after that time a blocked TCP port and the
> inability to opena TCP connection once caused my BMP monitor router
> doing the active open to crash (known and now fixed bug).
>
> > Thus adding this is a significant rework of BMP.
>
> I assume my desire for UDP above will never happen as a result.  Oh
> well.
>
> John
>
> ___
> GROW mailing list
> GROW@ietf.org
> https://www.ietf.org/mailman/listinfo/grow
>
___
GROW mailing list
GROW@ietf.org
https://www.ietf.org/mailman/listinfo/grow

Re: [GROW] is TCP the right layer for BMP session resumption?

[John Kristoff]

On Tue, 9 Mar 2021 20:44:18 +
"Jakob Heitz \(jheitz\)"  wrote:

> I've seen this session resumption technique in the '90s.
> BMP is a one-way protocol. The BMP server sends nothing.

I kind of wish my BMP router monitor was able to transport data over UDP
to the listening station like syslog and flow data.  I would have
especially liked this after that time a blocked TCP port and the
inability to opena TCP connection once caused my BMP monitor router
doing the active open to crash (known and now fixed bug).

> Thus adding this is a significant rework of BMP.

I assume my desire for UDP above will never happen as a result.  Oh
well.

John

___
GROW mailing list
GROW@ietf.org
https://www.ietf.org/mailman/listinfo/grow

Re: [GROW] is TCP the right layer for BMP session resumption?

[Jakob Heitz (jheitz)]

I've seen this session resumption technique in the '90s.
BMP is a one-way protocol. The BMP server sends nothing.
Thus adding this is a significant rework of BMP.
On the router, it will require remembering all the withdraws
that occurred in the BMP serial number window, so that window will
need to be limited. If the window exceeds its maximum, then
it would still be a hard reset.
If we do this, I advocate for a 64 bit serial number.

Regards,
Jakob.

-Original Message-
From: GROW  On Behalf Of Job Snijders
Sent: Tuesday, March 9, 2021 4:26 AM
To: draft-tppy-bmp-seamless-sess...@ietf.org; grow@ietf.org
Subject: [GROW] is TCP the right layer for BMP session resumption?

Dear group,

Yesterday we had the pleasure to hear a report from Thomas Graf on new
BMP work.

The https://datatracker.ietf.org/doc/html/draft-tppy-bmp-seamless-session-00
document outlines a concept to allow BMP clients to resume 'an existing'
session with the BMP server, reducing the need to re-transmit
information the client 'already knows'.

I informally polled some people with the question 'thoughts on
TCP_FAST_OPEN? It was brought to my attention a userspace server daemon
is not aware whether a TCP connection was set up with FAST_OPEN or not.

Furthermore, TCP_FAST_OPEN's primary use case appears to be to reduce
the burden of TCP Three-way handshake on 'short-lived' connections, and
was *not* designed for general purpose 'session resumption'.

RFC 7413 appears to suggest TCP_FAST_OPEN can be used to jam a 'small'
message (like a HTTP 302 response to a GET request) into the SYN-ACK,
whereas BMP Session Resumption is not about 'a quick and small reply',
but rather "previously there was lots of data, are you aware of that
previous data? By the way, what will follow next is lots and lots of
more data!".

TCP_FAST_OPEN appears a poor fit for the objective of BMP Seamless
Session Resumption.

If not TCP_FAST_OPEN, then what?


Perhaps inspiration can be taken from RFC 6810's RPKI-To-Router (RTR)
protocol which leverages a "Session ID" and "Serial Number" to allow
efficient (even transport-protocol agnostic!) session resumption.

This same style of session resumption mechanism can also be found in the
RPKI Repository Delta Protocol (RRDP).

Perhaps BMP would benefit from a similar resumption mechanism to reduce
the burden on servers & clients?

I welcome insights and feedback from the working group.

Kind regards,

Job

___
GROW mailing list
GROW@ietf.org
https://www.ietf.org/mailman/listinfo/grow

___
GROW mailing list
GROW@ietf.org
https://www.ietf.org/mailman/listinfo/grow

[GROW] is TCP the right layer for BMP session resumption?

[Job Snijders]

Dear group,

Yesterday we had the pleasure to hear a report from Thomas Graf on new
BMP work.

The https://datatracker.ietf.org/doc/html/draft-tppy-bmp-seamless-session-00
document outlines a concept to allow BMP clients to resume 'an existing'
session with the BMP server, reducing the need to re-transmit
information the client 'already knows'.

I informally polled some people with the question 'thoughts on
TCP_FAST_OPEN? It was brought to my attention a userspace server daemon
is not aware whether a TCP connection was set up with FAST_OPEN or not.

Furthermore, TCP_FAST_OPEN's primary use case appears to be to reduce
the burden of TCP Three-way handshake on 'short-lived' connections, and
was *not* designed for general purpose 'session resumption'.

RFC 7413 appears to suggest TCP_FAST_OPEN can be used to jam a 'small'
message (like a HTTP 302 response to a GET request) into the SYN-ACK,
whereas BMP Session Resumption is not about 'a quick and small reply',
but rather "previously there was lots of data, are you aware of that
previous data? By the way, what will follow next is lots and lots of
more data!".

TCP_FAST_OPEN appears a poor fit for the objective of BMP Seamless
Session Resumption.

If not TCP_FAST_OPEN, then what?


Perhaps inspiration can be taken from RFC 6810's RPKI-To-Router (RTR)
protocol which leverages a "Session ID" and "Serial Number" to allow
efficient (even transport-protocol agnostic!) session resumption.

This same style of session resumption mechanism can also be found in the
RPKI Repository Delta Protocol (RRDP).

Perhaps BMP would benefit from a similar resumption mechanism to reduce
the burden on servers & clients?

I welcome insights and feedback from the working group.

Kind regards,

Job

___
GROW mailing list
GROW@ietf.org
https://www.ietf.org/mailman/listinfo/grow