Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - draft-ietf-lsr-ospf-bfd-strict-mode-04

[Ketan Talaulikar]

Hi All,

The authors will work on and share the updated text to review with the WG.

Quickly want to point out that the draft does not bring any new
requirements in BFD for link quality monitoring; the stability issues that
we are discussing are related to BFD sessions going down due to
intermittent packet loss. If/when BFD WG brings any new enhancements,
protocols like OSPF will obviously leverage the same.

Thanks,
Ketan


On Tue, Feb 1, 2022 at 1:59 AM Jeffrey Haas  wrote:

> Les,
>
>
> On Jan 31, 2022, at 2:47 PM, Les Ginsberg (ginsberg) 
> wrote:
> I have not asked for BFD extensions.
> I have stated that “IF” additional functionality is required from BFD that
> the proper place to discuss that is in the BFD WG – and such discussions
> are definitely not in scope of this draft.
>
>
> Agreed.
>
>
> The main content of this lengthy thread is Robert asking for additional
> specification in this draft and other folks (myself, Albert, Ketan) saying
> it doesn’t belong in this draft. Which is why I agree with everything you
> say below except for your perception that you are agreeing with Robert. You
> are actually agreeing with myself, Albert, Ketan. 
>
>
> I was largely agreeing with Robert about the state of BFD and how it
> likely impacts the OSPF feature.  BFD isn't helpful beyond the Up/Down
> signaling, how it handles too much noise, etc.  So, I think we're all in
> agreement about these things.
>
> What the protocol chooses to do about the signaling BFD provides is up to
> the protocol. I think we're all agreeing that the strict mode helps with
> making sure implementations use the inputs to their state machines
> consistently.
>
> Whether there's any sort of "pause button" used by the implementation once
> the ordering is agreed upon seems to be where the disagreements have been.
> Such a pause button isn't appropriate for BFD.
>
> And now we're fully in sync. :-)
>
> -- Jeff
>
>
___
Lsr mailing list
Lsr@ietf.org
https://www.ietf.org/mailman/listinfo/lsr

Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - draft-ietf-lsr-ospf-bfd-strict-mode-04

[Ketan Talaulikar]

Hi Robert,

The comparison between the BFD strict-mode of operation proposals for OSPF
and BGP are not directly comparable since their FSMs are different. In
OSPF, the FSM is going to wait indefinitely in Init state until the BFD
session is set up, while that is not the case for BGP and hence BGP needs
the "BGP BFD Hold time" to close/restart the neighbor FSM.

Please correct me if I am reading this wrong.

What the OSPF draft discusses in Sec 5 is a "hold-down" wait period where
even though the BFD session is established the protocol FSM does not
proceed further until a period of time has passed to ensure the stability
of the BFD session.

Thanks,
Ketan


On Mon, Jan 31, 2022 at 8:29 PM Robert Raszuk  wrote:

> Les & Ketan
>
>
>> Nowadays, it is also common to see the "break-in-middle" failures. we use
>> BFD to detect this sort of failure within sub-second. And to dampen this
>> sort of break-in-middle failures, we will need to use BFD
>> holdtime/dampening.
>>
>
> Another data point to the above and this discussion which Albert is
> co-author of.
>
> Ref:
> https://datatracker.ietf.org/doc/html/draft-ietf-idr-bgp-bfd-strict-mode
>
> Please see the below paragraph which clearly says *BGP BFD Hold time*:
>
>If the BFD session does not transition to the Up state, and the
>HoldTimer has been negotiated to a non-zero value, the BGP FSM will
>close the session appropriately.  If the HoldTimer has been
>negotiated to a zero value, the session should be closed after a time
>of X.  This time X is referred as "BGP BFD Hold time".  The proposed
>default BGP BFD Hold time value is 30 seconds.  The BGP BFD Hold time
>value is configurable.
>
> To me it is clear that BGP BFD Hold time is on the client side and here
> affects BGP FSM.
>
> Thx,
> Robert.
>
>
>
>
>
>
>
> From: ginsb...@cisco.com At: 01/30/22 14:38:37 UTC-5:00
>> To: rob...@raszuk.net, ketant.i...@gmail.com
>> Cc: Albert Fu (BLOOMBERG/ 120 PARK ) ,
>> a...@cisco.com, draft-ietf-lsr-ospf-bfd-strict-m...@ietf.org,
>> lsr@ietf.org
>> Subject: RE: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD"
>> - draft-ietf-lsr-ospf-bfd-strict-mode-04
>>
>> Robert –
>>
>>
>>
>> Here is what you said (emphasis added):
>>
>>
>>
>> 
>>
>> But the timer I am suggesting is not related to BFD operation, but to
>> OSPF (and/or ISIS). It is not about BFD sessions being UP or DOWN. It is
>> about *allowing BFD for more testing (with various parameters (for
>> example increasing test packet size in some discrete steps)* before OSPF
>> is happy to bring the adj. up.
>>
>> 
>>
>>
>>
>> Point #1: If you want BFD to do more testing (such as MTU testing) then
>> clearly you need extensions to BFD (such as
>> https://datatracker.ietf.org/doc/draft-ietf-bfd-large-packets/ )
>>
>>
>>
>> Point #2: The existing timers (as Ketan points out are mentioned in
>> Section 5) are applied today at the OSPF level precisely because OSPF does
>> not currently have strict-mode operation. So in a flapping scenario you
>> could see the following behavior:
>>
>>
>>
>> a)BFD goes down
>>
>> b)OSPF goes down in response to BFD
>>
>> c)OSPF comes back up
>>
>> d)Link is still unstable – so traffic is being dropped some of the time –
>> but perhaps OSPF adjacency stays up (i.e., OSPF hellos get through often
>> enough to keep the OSPF adjacency up)
>>
>>
>>
>> So some implementations have chosen to insert a delay following “b”. This
>> doesn’t guarantee stability, but hopefully makes it less likely. And
>> because OSPF today does NOT wait for BFD to come up, the delay has to be
>> implemented at the OSPF level.
>>
>>
>>
>> Once you have strict mode support, the sequence becomes:
>>
>>
>>
>> a)BFD goes down
>>
>> b)OSPF goes down in response to BFD
>>
>> c)BFD comes back up
>>
>> d)OSPF comes back up
>>
>>
>>
>> Now, if the concern is that BFD comes back up while the link is still
>> unstable, the way to address that is to put a delay either before BFD
>> attempts to bring up a new session or a delay after achieving UP state
>> before it signals UP to its clients – such as OSPF. This is a better
>> solution because all BFD clients benefit from this. Ad if the link is still
>> unstable, it is more likely that the BFD session will go down during the
>> delay period than it would be for OSPF because the BFD timers are
>> significantly more aggressive.
>>
>> (BTW, this behavior can be done w/o a BFD protocol extension – it is
>> purely an implementation choice.)
>>
>>
>>
>> From a design perspective, dampening is always best done at the lowest
>> layer possible. In most cases, interface layer dampening is best. If that
>> is not reliable for some reason, then move one layer up – not two layers up.
>>
>>
>>
>>Les
>>
>>
>>
>>
>>
>> *From:* Robert Raszuk 
>> *Sent:* Sunday, January 30, 2022 10:05 AM
>> *To:* Ketan Talaulikar 
>> *Cc:* Les Ginsberg (ginsberg) ; Acee Lindem (acee) <
>> a...@cisco.com>; draft-ietf-lsr-ospf-bfd-strict-m...@ietf.org; Albert Fu
>> ; lsr 
>> 

Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - draft-ietf-lsr-ospf-bfd-strict-mode-04

[Jeffrey Haas]

Les,


> On Jan 31, 2022, at 2:47 PM, Les Ginsberg (ginsberg)  
> wrote:
> I have not asked for BFD extensions.
> I have stated that “IF” additional functionality is required from BFD that 
> the proper place to discuss that is in the BFD WG – and such discussions are 
> definitely not in scope of this draft.

Agreed.

>  
> The main content of this lengthy thread is Robert asking for additional 
> specification in this draft and other folks (myself, Albert, Ketan) saying it 
> doesn’t belong in this draft. Which is why I agree with everything you say 
> below except for your perception that you are agreeing with Robert. You are 
> actually agreeing with myself, Albert, Ketan. 

I was largely agreeing with Robert about the state of BFD and how it likely 
impacts the OSPF feature.  BFD isn't helpful beyond the Up/Down signaling, how 
it handles too much noise, etc.  So, I think we're all in agreement about these 
things.

What the protocol chooses to do about the signaling BFD provides is up to the 
protocol. I think we're all agreeing that the strict mode helps with making 
sure implementations use the inputs to their state machines consistently.

Whether there's any sort of "pause button" used by the implementation once the 
ordering is agreed upon seems to be where the disagreements have been.  Such a 
pause button isn't appropriate for BFD.  

And now we're fully in sync. :-)

-- Jeff

___
Lsr mailing list
Lsr@ietf.org
https://www.ietf.org/mailman/listinfo/lsr

Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - draft-ietf-lsr-ospf-bfd-strict-mode-04

[Robert Raszuk]

Hey Albert,

Ok now we are in sync as far as what is the topic.

I think such a delay is very useful and completely in the spirit of OSPF or
ISIS strict-mode operation.

So I do recommend that the draft should discuss it.

The default can be 0 if you think that is the proper value. But the
operational section of that draft IMO is exactly the place where such
paragraph should be placed. Maybe I would not be so persistent in this
little thread if Les wouldn't indicate that current timer will be removed
(current timer acting as an artificial delay and being much longer then
time needed for BFD to come UP).

Author's choice. My mission is accomplished here for the WG mailing list
records :)

Cheers,
Robert.



On Mon, Jan 31, 2022 at 9:04 PM Albert Fu (BLOOMBERG/ 120 PARK) <
af...@bloomberg.net> wrote:

> Hi Robert,
>
> As mentioned in my previous email, I feel it is better not to specify in
> the draft the timer for when OSPF should come up after BFD is up.
>
> The current implementation is for OSPF to come up as soon as BFD is up. A
> user can change this behaviour via configuration, to delay when OSPF can
> come up after BFD is up. Different customers may have different delay
> requirements, and there may also be platform dependent limitation.
>
> Thanks
>
> Albert
>
> From: rob...@raszuk.net At: 01/31/22 14:52:43 UTC-5:00
> To: Albert Fu (BLOOMBERG/ 120 PARK ) 
> Cc: a...@cisco.com, draft-ietf-lsr-ospf-bfd-strict-m...@ietf.org,
> ginsb...@cisco.com, ketant.i...@gmail.com, lsr@ietf.org
> Subject: Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD"
> - draft-ietf-lsr-ospf-bfd-strict-mode-04
>
> Hi Albert,
>
> On Mon, Jan 31, 2022 at 8:38 PM Albert Fu (BLOOMBERG/ 120 PARK) <
> af...@bloomberg.net> wrote:
>
>> Hi Robert,
>>
>> Do you mean we should make it mandatory in the draft to stipulate a delay
>> time between when OSPF should wait for BFD to come up?
>>
>
> No.
>
> The timer is for OSPF to bring adj up only after X timer expires from the
> moment BFD session came up and stayed up (never went down).
>
> No changes to BFD needed at all.
>
> Trivial to implement on the client side and very useful operationally.
>
> Thx,
> Robert
>
>
___
Lsr mailing list
Lsr@ietf.org
https://www.ietf.org/mailman/listinfo/lsr

Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - draft-ietf-lsr-ospf-bfd-strict-mode-04

[Les Ginsberg (ginsberg)]

> I have stated that “IF” additional functionality is required from BFD

No one says so,
[LES:] Good. Can we end this thread then?
[
   Les
___
Lsr mailing list
Lsr@ietf.org
https://www.ietf.org/mailman/listinfo/lsr

Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - draft-ietf-lsr-ospf-bfd-strict-mode-04

[Albert Fu (BLOOMBERG/ 120 PARK)]

Hi Robert,

As mentioned in my previous email, I feel it is better not to specify in the 
draft the timer for when OSPF should come up after BFD is up. 

The current implementation is for OSPF to come up as soon as BFD is up. A user 
can change this behaviour via configuration, to delay when OSPF can come up 
after BFD is up. Different customers may have different delay requirements, and 
there may also be platform dependent limitation.

Thanks

Albert

From: rob...@raszuk.net At: 01/31/22 14:52:43 UTC-5:00To:  Albert Fu 
(BLOOMBERG/ 120 PARK ) 
Cc:  a...@cisco.com,  draft-ietf-lsr-ospf-bfd-strict-m...@ietf.org,  
ginsb...@cisco.com,  ketant.i...@gmail.com,  lsr@ietf.org
Subject: Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - 
draft-ietf-lsr-ospf-bfd-strict-mode-04

Hi Albert,
On Mon, Jan 31, 2022 at 8:38 PM Albert Fu (BLOOMBERG/ 120 PARK) 
 wrote:

Hi Robert,

Do you mean we should make it mandatory in the draft to stipulate a delay time 
between when OSPF should wait for BFD to come up?

No. 

The timer is for OSPF to bring adj up only after X timer expires from the 
moment BFD session came up and stayed up (never went down). 

No changes to BFD needed at all. 

Trivial to implement on the client side and very useful operationally. 

Thx,
Robert


 
I don't know how others feel, but I tend to agree the main author of this 
Draft, Ketan, that it is best to leave the delay timer out of this draft. 

There is already an implicit understanding that BFD must be up before OSPF can 
progress to the adjacency phase.

And I can think of deployments with many redundant links where the delay can be 
large value, and some scenario say sites with only 1 redundant link where it is 
not desirable for the delay not to be too lengthy, to avoid both links being 
down at the same time and cutting communication to the site completely.

I have also tested current implementations where the delays do not have to 
match (e.g. one side with delay, and one side no delay).

IMO, it is better not to make the delay a part of the standard.

Thanks

Albert


From: rob...@raszuk.net At: 01/31/22 13:51:56 UTC-5:00To:  Albert Fu 
(BLOOMBERG/ 120 PARK ) 
Cc:  ginsb...@cisco.com,  ketant.i...@gmail.com,  a...@cisco.com,  
draft-ietf-lsr-ospf-bfd-strict-m...@ietf.org,  lsr@ietf.org
Subject: Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - 
draft-ietf-lsr-ospf-bfd-strict-mode-04

Albert,

> It serves as a sanity check that there's indeed a working 
> BFD for a period of time before OSPF adjacency is allowed 
> to progress.

And that is precisely what I am suggesting that should be both mandatory and 
part of this draft. Not an optional nice to have vendor knob. 

Clearly it does not belong to BFD spec or WG what Les and Ketan are trying to 
suggest. 

Regards,
R.


On Mon, Jan 31, 2022 at 6:52 PM Albert Fu (BLOOMBERG/ 120 PARK) 
 wrote:

Hi Robert,

The BGP BFD hold-time mentioned in the BGP BFD strict mode draft has different 
meaning from the holdtime/delay/dampening that has been discussed in this forum 
thus far. 

The BGP BFD hold-time, as per the BGP BFD draft below, is user configurable, 
and is used to bring down the BGP session if BFD session is not established 
within default of 30s, when the negotiated "BGP HoldTimer" is 0. 


The OSPF hold-time/delay/dampening that we have been discussing so far is the 
delay from when BFD comes up to when OSPF will be allowed to come up. This, as 
Ketan mentioned, is outside the scope of this draft. 

In my testing with both Cisco and Juniper implementation, the OSPF 
hold-time/delay/dampening timers are quite arbitrary. You could have no delay 
(which means bring up OSPF asap), or have it configured on one side only. It 
serves as a sanity check that there's indeed a working BFD for a period of time 
before OSPF adjacency is allowed to progress.

Thanks

Albert

From: rob...@raszuk.net At: 01/31/22 09:59:48 UTC-5:00To:  Albert Fu 
(BLOOMBERG/ 120 PARK ) ,  ginsb...@cisco.com,  ketant.i...@gmail.com
Cc:  a...@cisco.com,  draft-ietf-lsr-ospf-bfd-strict-m...@ietf.org,  
lsr@ietf.org
Subject: Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - 
draft-ietf-lsr-ospf-bfd-strict-mode-04

Les & Ketan
 
Nowadays, it is also common to see the "break-in-middle" failures. we use BFD 
to detect this sort of failure within sub-second. And to dampen this sort of 
break-in-middle failures, we will need to use BFD holdtime/dampening. 


Another data point to the above and this discussion which Albert is co-author 
of. 

Ref: https://datatracker.ietf.org/doc/html/draft-ietf-idr-bgp-bfd-strict-mode

Please see the below paragraph which clearly says BGP BFD Hold time: 

   If the BFD session does not transition to the Up state, and the
   HoldTimer has been negotiated to a non-zero value, the BGP FSM will
   close the session appropriately.  If the HoldTimer has been
   negotiated to a zero value, the session should be closed after a time
   of X.  This time X 

Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - draft-ietf-lsr-ospf-bfd-strict-mode-04

[Robert Raszuk]

> I have stated that “IF” additional functionality is required from BFD

No one says so, It would be not realistic to require for BFD to come up in
a hidden mode, operate for timer X then when timer X expires signal that to
clients. And this is precisely what you are suggesting as a push back.

It is client thing to delay their action according to the operational
needs.

Many thx,
R.

On Mon, Jan 31, 2022 at 8:48 PM Les Ginsberg (ginsberg) 
wrote:

> Jeff –
>
>
>
> I appreciate that you have been pulled into reading a very lengthy thread
> and then commenting  on it – which is a difficult/time consuming  thing to
> do accurately.
>
> And I certainly welcome your input and agree with your input.
>
>
>
> I have not asked for BFD extensions.
>
> I have stated that “IF” additional functionality is required from BFD that
> the proper place to discuss that is in the BFD WG – and such discussions
> are definitely not in scope of this draft.
>
>
>
> The main content of this lengthy thread is Robert asking for additional
> specification in this draft and other folks (myself, Albert, Ketan) saying
> it doesn’t belong in this draft. Which is why I agree with everything you
> say below except for your perception that you are agreeing with Robert. You
> are actually agreeing with myself, Albert, Ketan. 
>
>
>
> Thanx for your participation.
>
>
>
> Les
>
>
>
> *From:* Jeffrey Haas 
> *Sent:* Monday, January 31, 2022 11:28 AM
> *To:* Robert Raszuk 
> *Cc:* Ketan Talaulikar ; Les Ginsberg (ginsberg) <
> ginsb...@cisco.com>; draft-ietf-lsr-ospf-bfd-strict-m...@ietf.org; Acee
> Lindem (acee) ; Albert Fu ; lsr <
> lsr@ietf.org>
> *Subject:* Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for
> BFD" - draft-ietf-lsr-ospf-bfd-strict-mode-04
>
>
>
> [Note that I read the LSR mailing list infrequently, but this thread was
> brought to my attention.]
>
>
>
> I wish to largely support Robert's point here.  BFD is not intended as a
> link quality protocol.  It's a very simple hello protocol that can operate
> quite quickly and provide simple edge transition events of Up and Down.
>
>
>
> There has been work in the BFD Working Group over the years to attempt to
> bring more of "link quality" behaviors to the protocol.  One, of interest
> to this thread, is the BFD for Large Packets work, which can support MTU
> probes as part of BFD operation.
>
>
>
> draft-ietf-bfd-stability discusses leveraging BFD internal state to help
> look at link instability issues as BFD sees them.
>
>
>
> And, of course, Greg Mirsky had several times he wanted to get BFD to do
> more active behaviors.  He was encouraged to leverage the BFD machinery in
> his own non-BFD draft if he found it helpful.  I suspect he'll respond to
> this thread with comments on his thinking here.
>
>
>
> That said, the BFD strict work is about removing control-plane protocol
> ambiguity with regards to how it uses BFD and how the state machines
> interact with each other.  I think that work has been reasonably done.
>
>
>
> The thing that BFD isn't about in such contexts is being more than a
> simple proxy for the link being of bad enough quality for BFD to go down
> taking the client protocols down with it.  It's important for those client
> protocols and the operators to set the timers and Detection Multiplier
> (number of lost packets) to speeds they think support their needs.  If you
> have a noisy link that can drop several packets in succession and that's
> what you want to be your trigger, BFD is your protocol.  If you want it to
> take an apparently continuous loss over most of a second, BFD can do that
> too if you tune your timers appropriately.
>
>
>
> But, as you say Robert, it's not intended to be a general IPPM style
> tool.  I don't believe the BFD strict drafts should try to treat BFD as if
> it is one.
>
>
>
> -- Jeff
>
>
>
>
>
>
>
> On Jan 31, 2022, at 5:31 AM, Robert Raszuk  wrote:
>
>
>
> HI Ketan & Les,
>
>
>
> To finish this topic I would like to observe that IMHO you have it quite
> backwords.
>
>
>
> *Comment #1*
>
>
>
> The tone of your expressions is trying to illustrate that there can be
> many clients for given link probing tool (here BFD). In reality the
> situation is vastly different. There is usually one link state IGP running
> on the node and given set of probing protocols are associated with it.
> Moreover, the world does not end on BFD. BFD is just one possible tool, but
> more and more path probing tools are emerging or are already deployed.
> Asking for each of them to introduce into their state machine a new
> behaviour to delay reporting UP state on a per client basis is nothing else
> then just pushing the problem aways and not caring for the cost associated
> with it.
>
>
>
> *Comment #2 *
>
>
>
> BFD is a great tool to tell you if the end to end path is UP or DOWN. It
> was not designed to give you any characteristics or metrics for the path
> quality.
>
>
>
> So all assertions of that notion in your draft are simply 

Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - draft-ietf-lsr-ospf-bfd-strict-mode-04

[Robert Raszuk]

Hi Albert,

On Mon, Jan 31, 2022 at 8:38 PM Albert Fu (BLOOMBERG/ 120 PARK) <
af...@bloomberg.net> wrote:

> Hi Robert,
>
> Do you mean we should make it mandatory in the draft to stipulate a delay
> time between when OSPF should wait for BFD to come up?
>

No.

The timer is for OSPF to bring adj up only after X timer expires from the
moment BFD session came up and stayed up (never went down).

No changes to BFD needed at all.

Trivial to implement on the client side and very useful operationally.

Thx,
Robert




> I don't know how others feel, but I tend to agree the main author of this
> Draft, Ketan, that it is best to leave the delay timer out of this draft.
>
> There is already an implicit understanding that BFD must be up before OSPF
> can progress to the adjacency phase.
>
> And I can think of deployments with many redundant links where the delay
> can be large value, and some scenario say sites with only 1 redundant link
> where it is not desirable for the delay not to be too lengthy, to avoid
> both links being down at the same time and cutting communication to the
> site completely.
>
> I have also tested current implementations where the delays do not have to
> match (e.g. one side with delay, and one side no delay).
>
> IMO, it is better not to make the delay a part of the standard.
>
> Thanks
>
> Albert
>
>
> From: rob...@raszuk.net At: 01/31/22 13:51:56 UTC-5:00
> To: Albert Fu (BLOOMBERG/ 120 PARK ) 
> Cc: ginsb...@cisco.com, ketant.i...@gmail.com, a...@cisco.com,
> draft-ietf-lsr-ospf-bfd-strict-m...@ietf.org, lsr@ietf.org
> Subject: Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD"
> - draft-ietf-lsr-ospf-bfd-strict-mode-04
>
> Albert,
>
> > It serves as a sanity check that there's indeed a working
> > BFD for a period of time before OSPF adjacency is allowed
> > to progress.
>
> And that is precisely what I am suggesting that should be both mandatory
> and part of this draft. Not an optional nice to have vendor knob.
>
> Clearly it does not belong to BFD spec or WG what Les and Ketan are trying
> to suggest.
>
> Regards,
> R.
>
>
>
>
>
>
>
>
> On Mon, Jan 31, 2022 at 6:52 PM Albert Fu (BLOOMBERG/ 120 PARK) <
> af...@bloomberg.net> wrote:
>
>> Hi Robert,
>>
>> The BGP BFD hold-time mentioned in the BGP BFD strict mode draft has
>> different meaning from the holdtime/delay/dampening that has been discussed
>> in this forum thus far.
>>
>> The BGP BFD hold-time, as per the BGP BFD draft below, is user
>> configurable, and is used to bring down the BGP session if BFD session is
>> not established within default of 30s, when the negotiated "BGP HoldTimer"
>> is 0.
>>
>>
>> The OSPF hold-time/delay/dampening that we have been discussing so far is
>> the delay from when BFD comes up to when OSPF will be allowed to come up.
>> This, as Ketan mentioned, is outside the scope of this draft.
>>
>> In my testing with both Cisco and Juniper implementation, the OSPF
>> hold-time/delay/dampening timers are quite arbitrary. You could have no
>> delay (which means bring up OSPF asap), or have it configured on one side
>> only. It serves as a sanity check that there's indeed a working BFD for a
>> period of time before OSPF adjacency is allowed to progress.
>>
>> Thanks
>>
>> Albert
>>
>> From: rob...@raszuk.net At: 01/31/22 09:59:48 UTC-5:00
>> To: Albert Fu (BLOOMBERG/ 120 PARK ) ,
>> ginsb...@cisco.com, ketant.i...@gmail.com
>> Cc: a...@cisco.com, draft-ietf-lsr-ospf-bfd-strict-m...@ietf.org,
>> lsr@ietf.org
>> Subject: Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD"
>> - draft-ietf-lsr-ospf-bfd-strict-mode-04
>>
>> Les & Ketan
>>
>>
>>> Nowadays, it is also common to see the "break-in-middle" failures. we
>>> use BFD to detect this sort of failure within sub-second. And to dampen
>>> this sort of break-in-middle failures, we will need to use BFD
>>> holdtime/dampening.
>>>
>>
>> Another data point to the above and this discussion which Albert is
>> co-author of.
>>
>> Ref:
>> https://datatracker.ietf.org/doc/html/draft-ietf-idr-bgp-bfd-strict-mode
>>
>> Please see the below paragraph which clearly says *BGP BFD Hold time*:
>>
>>If the BFD session does not transition to the Up state, and the
>>HoldTimer has been negotiated to a non-zero value, the BGP FSM will
>>close the session appropriately.  If the HoldTimer has been
>>negotiated to a zero value, the session should be closed after a time
>>of X.  This time X is referred as "BGP BFD Hold time".  The proposed
>>default BGP BFD Hold time value is 30 seconds.  The BGP BFD Hold time
>>value is configurable.
>>
>> To me it is clear that BGP BFD Hold time is on the client side and here
>> affects BGP FSM.
>>
>> Thx,
>> Robert.
>>
>>
>>
>>
>>
>>
>>
>> From: ginsb...@cisco.com At: 01/30/22 14:38:37 UTC-5:00
>>> To: rob...@raszuk.net, ketant.i...@gmail.com
>>> Cc: Albert Fu (BLOOMBERG/ 120 PARK ) ,
>>> a...@cisco.com, draft-ietf-lsr-ospf-bfd-strict-m...@ietf.org,
>>> lsr@ietf.org
>>> Subject: RE: 

Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - draft-ietf-lsr-ospf-bfd-strict-mode-04

[Les Ginsberg (ginsberg)]

Jeff –

I appreciate that you have been pulled into reading a very lengthy thread and 
then commenting  on it – which is a difficult/time consuming  thing to do 
accurately.
And I certainly welcome your input and agree with your input.

I have not asked for BFD extensions.
I have stated that “IF” additional functionality is required from BFD that the 
proper place to discuss that is in the BFD WG – and such discussions are 
definitely not in scope of this draft.

The main content of this lengthy thread is Robert asking for additional 
specification in this draft and other folks (myself, Albert, Ketan) saying it 
doesn’t belong in this draft. Which is why I agree with everything you say 
below except for your perception that you are agreeing with Robert. You are 
actually agreeing with myself, Albert, Ketan. 

Thanx for your participation.

Les

From: Jeffrey Haas 
Sent: Monday, January 31, 2022 11:28 AM
To: Robert Raszuk 
Cc: Ketan Talaulikar ; Les Ginsberg (ginsberg) 
; draft-ietf-lsr-ospf-bfd-strict-m...@ietf.org; Acee Lindem 
(acee) ; Albert Fu ; lsr 
Subject: Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - 
draft-ietf-lsr-ospf-bfd-strict-mode-04

[Note that I read the LSR mailing list infrequently, but this thread was 
brought to my attention.]

I wish to largely support Robert's point here.  BFD is not intended as a link 
quality protocol.  It's a very simple hello protocol that can operate quite 
quickly and provide simple edge transition events of Up and Down.

There has been work in the BFD Working Group over the years to attempt to bring 
more of "link quality" behaviors to the protocol.  One, of interest to this 
thread, is the BFD for Large Packets work, which can support MTU probes as part 
of BFD operation.

draft-ietf-bfd-stability discusses leveraging BFD internal state to help look 
at link instability issues as BFD sees them.

And, of course, Greg Mirsky had several times he wanted to get BFD to do more 
active behaviors.  He was encouraged to leverage the BFD machinery in his own 
non-BFD draft if he found it helpful.  I suspect he'll respond to this thread 
with comments on his thinking here.

That said, the BFD strict work is about removing control-plane protocol 
ambiguity with regards to how it uses BFD and how the state machines interact 
with each other.  I think that work has been reasonably done.

The thing that BFD isn't about in such contexts is being more than a simple 
proxy for the link being of bad enough quality for BFD to go down taking the 
client protocols down with it.  It's important for those client protocols and 
the operators to set the timers and Detection Multiplier (number of lost 
packets) to speeds they think support their needs.  If you have a noisy link 
that can drop several packets in succession and that's what you want to be your 
trigger, BFD is your protocol.  If you want it to take an apparently continuous 
loss over most of a second, BFD can do that too if you tune your timers 
appropriately.

But, as you say Robert, it's not intended to be a general IPPM style tool.  I 
don't believe the BFD strict drafts should try to treat BFD as if it is one.

-- Jeff




On Jan 31, 2022, at 5:31 AM, Robert Raszuk 
mailto:rob...@raszuk.net>> wrote:

HI Ketan & Les,

To finish this topic I would like to observe that IMHO you have it quite 
backwords.

Comment #1

The tone of your expressions is trying to illustrate that there can be many 
clients for given link probing tool (here BFD). In reality the situation is 
vastly different. There is usually one link state IGP running on the node and 
given set of probing protocols are associated with it. Moreover, the world does 
not end on BFD. BFD is just one possible tool, but more and more path probing 
tools are emerging or are already deployed. Asking for each of them to 
introduce into their state machine a new behaviour to delay reporting UP state 
on a per client basis is nothing else then just pushing the problem aways and 
not caring for the cost associated with it.

Comment #2

BFD is a great tool to tell you if the end to end path is UP or DOWN. It was 
not designed to give you any characteristics or metrics for the path quality.

So all assertions of that notion in your draft are simply wrong. While sure 
there are proposals to extend BFD probe packets with arbitrary large payload to 
tell you if at some packet size you can still reach the other end they are 
still nothing close to measure any form of link performance or detect "A 
degraded or poor quality link"

Thx a lot,
R.

On Mon, Jan 31, 2022 at 5:48 AM Ketan Talaulikar 
mailto:ketant.i...@gmail.com>> wrote:
Hi Les,

I agree with you that mechanisms like dampening and hold-down are best achieved 
at the lowest levels (in this case in the monitoring protocol like BFD) instead 
of in each routing protocol on the top.

Now whether this means we include/support the signaling of the parameters for 
these mechanisms in BFD or whether 

Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - draft-ietf-lsr-ospf-bfd-strict-mode-04

[Albert Fu (BLOOMBERG/ 120 PARK)]

Hi Robert,

Do you mean we should make it mandatory in the draft to stipulate a delay time 
between when OSPF should wait for BFD to come up?

I don't know how others feel, but I tend to agree the main author of this 
Draft, Ketan, that it is best to leave the delay timer out of this draft. 

There is already an implicit understanding that BFD must be up before OSPF can 
progress to the adjacency phase.

And I can think of deployments with many redundant links where the delay can be 
large value, and some scenario say sites with only 1 redundant link where it is 
not desirable for the delay not to be too lengthy, to avoid both links being 
down at the same time and cutting communication to the site completely.

I have also tested current implementations where the delays do not have to 
match (e.g. one side with delay, and one side no delay).

IMO, it is better not to make the delay a part of the standard.

Thanks

Albert


From: rob...@raszuk.net At: 01/31/22 13:51:56 UTC-5:00To:  Albert Fu 
(BLOOMBERG/ 120 PARK ) 
Cc:  ginsb...@cisco.com,  ketant.i...@gmail.com,  a...@cisco.com,  
draft-ietf-lsr-ospf-bfd-strict-m...@ietf.org,  lsr@ietf.org
Subject: Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - 
draft-ietf-lsr-ospf-bfd-strict-mode-04

Albert,

> It serves as a sanity check that there's indeed a working 
> BFD for a period of time before OSPF adjacency is allowed 
> to progress.

And that is precisely what I am suggesting that should be both mandatory and 
part of this draft. Not an optional nice to have vendor knob. 

Clearly it does not belong to BFD spec or WG what Les and Ketan are trying to 
suggest. 

Regards,
R.


On Mon, Jan 31, 2022 at 6:52 PM Albert Fu (BLOOMBERG/ 120 PARK) 
 wrote:

Hi Robert,

The BGP BFD hold-time mentioned in the BGP BFD strict mode draft has different 
meaning from the holdtime/delay/dampening that has been discussed in this forum 
thus far. 

The BGP BFD hold-time, as per the BGP BFD draft below, is user configurable, 
and is used to bring down the BGP session if BFD session is not established 
within default of 30s, when the negotiated "BGP HoldTimer" is 0. 


The OSPF hold-time/delay/dampening that we have been discussing so far is the 
delay from when BFD comes up to when OSPF will be allowed to come up. This, as 
Ketan mentioned, is outside the scope of this draft. 

In my testing with both Cisco and Juniper implementation, the OSPF 
hold-time/delay/dampening timers are quite arbitrary. You could have no delay 
(which means bring up OSPF asap), or have it configured on one side only. It 
serves as a sanity check that there's indeed a working BFD for a period of time 
before OSPF adjacency is allowed to progress.

Thanks

Albert

From: rob...@raszuk.net At: 01/31/22 09:59:48 UTC-5:00To:  Albert Fu 
(BLOOMBERG/ 120 PARK ) ,  ginsb...@cisco.com,  ketant.i...@gmail.com
Cc:  a...@cisco.com,  draft-ietf-lsr-ospf-bfd-strict-m...@ietf.org,  
lsr@ietf.org
Subject: Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - 
draft-ietf-lsr-ospf-bfd-strict-mode-04

Les & Ketan
 
Nowadays, it is also common to see the "break-in-middle" failures. we use BFD 
to detect this sort of failure within sub-second. And to dampen this sort of 
break-in-middle failures, we will need to use BFD holdtime/dampening. 


Another data point to the above and this discussion which Albert is co-author 
of. 

Ref: https://datatracker.ietf.org/doc/html/draft-ietf-idr-bgp-bfd-strict-mode

Please see the below paragraph which clearly says BGP BFD Hold time: 

   If the BFD session does not transition to the Up state, and the
   HoldTimer has been negotiated to a non-zero value, the BGP FSM will
   close the session appropriately.  If the HoldTimer has been
   negotiated to a zero value, the session should be closed after a time
   of X.  This time X is referred as "BGP BFD Hold time".  The proposed
   default BGP BFD Hold time value is 30 seconds.  The BGP BFD Hold time
   value is configurable.

To me it is clear that BGP BFD Hold time is on the client side and here affects 
BGP FSM. 

Thx,
Robert.


From: ginsb...@cisco.com At: 01/30/22 14:38:37 UTC-5:00To:  rob...@raszuk.net,  
ketant.i...@gmail.com
Cc:  Albert Fu (BLOOMBERG/ 120 PARK ) ,  a...@cisco.com,  
draft-ietf-lsr-ospf-bfd-strict-m...@ietf.org,  lsr@ietf.org
Subject: RE: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - 
draft-ietf-lsr-ospf-bfd-strict-mode-04

 

Robert – 
  
Here is what you said (emphasis added): 
  
 
But the timer I am suggesting is not related to BFD operation, but to OSPF 
(and/or ISIS). It is not about BFD sessions being UP or DOWN. It is about 
allowing BFD for more testing (with various parameters (for example increasing 
test packet size in some discrete steps) before OSPF is happy to bring the adj. 
up. 
 
  
Point #1: If you want BFD to do more testing (such as MTU testing) then clearly 
you need extensions to BFD (such as 

Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - draft-ietf-lsr-ospf-bfd-strict-mode-04

[Robert Raszuk]

> what new signal should BFD send to OSPF when this is done?

None. Lack of DOWN signal is enough for OSPF to proceed.

Thx,
R.

On Mon, Jan 31, 2022 at 6:50 PM Les Ginsberg (ginsberg) 
wrote:

> Robert –
>
>
>
> The paragraph you quote below has to do with BGP behavior in the event
> “BFD session does not transition to the Up state”.
>
> There is no disagreement about what the protocol (BGP or OSPF) should do
> in this case. The point of strict-mode is to “wait-for-BFD”.
>
>
>
> You, however, are trying to introduce some additional requirements. To
> this end you said:
>
>
>
> *“What I find missing in the draft is a mutually (between OSPF peers)
> timer fired after BFD session is up which in OSPF could hold on allowing
> BFD to do some more testing before declaring adj to be established. I think
> just bringing OSPF adj immediately after the BFD session is up is not a
> good thing.”*
>
>
>
> So apparently you want BFD to signal UP – but have the protocol do nothing
> until BFD completes some additional testing. What then was the point of BFD
> signaling UP to OSPF? And since you want the additional testing to be done
> by BFD, what new signal should BFD send to OSPF when this is done?
>
> The point of BFD sending UP to its clients is to indicate that BFD thinks
> the link has been verified from the BFD perspective. I do not see the point
> of sending two such signals. If you think current BFD testing is inadequate
> please ask for extensions to BFD (in the BFD WG).
>
>
>
> You also said:
>
>
>
> “BFD is a great tool to tell you if the end to end path is UP or DOWN. It
> was not designed to give you any characteristics or metrics for the path
> quality.”
>
>
>
> I agree. But if you are now proposing that protocol adjacencies should not
> come up until certain link quality metrics are met (e.g., link loss, delay)
> – you are moving into an area that is completely out of scope of this draft.
>
> I won’t dig deeper into what could be a very lengthy discussion. If you
> really want to pursue this idea, I suggest you write a new draft.
>
>
>
>Les
>
>
>
> *From:* Robert Raszuk 
> *Sent:* Monday, January 31, 2022 6:59 AM
> *To:* Albert Fu ; Les Ginsberg (ginsberg) <
> ginsb...@cisco.com>; Ketan Talaulikar 
> *Cc:* Acee Lindem (acee) ;
> draft-ietf-lsr-ospf-bfd-strict-m...@ietf.org; lsr 
> *Subject:* Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for
> BFD" - draft-ietf-lsr-ospf-bfd-strict-mode-04
>
>
>
> Les & Ketan
>
>
>
> Nowadays, it is also common to see the "break-in-middle" failures. we use
> BFD to detect this sort of failure within sub-second. And to dampen this
> sort of break-in-middle failures, we will need to use BFD
> holdtime/dampening.
>
>
>
> Another data point to the above and this discussion which Albert is
> co-author of.
>
>
>
> Ref:
> https://datatracker.ietf.org/doc/html/draft-ietf-idr-bgp-bfd-strict-mode
>
>
>
> Please see the below paragraph which clearly says *BGP BFD Hold time*:
>
>
>
>If the BFD session does not transition to the Up state, and the
>HoldTimer has been negotiated to a non-zero value, the BGP FSM will
>close the session appropriately.  If the HoldTimer has been
>negotiated to a zero value, the session should be closed after a time
>of X.  This time X is referred as "BGP BFD Hold time".  The proposed
>default BGP BFD Hold time value is 30 seconds.  The BGP BFD Hold time
>value is configurable.
>
>
>
> To me it is clear that BGP BFD Hold time is on the client side and here
> affects BGP FSM.
>
>
>
> Thx,
>
> Robert.
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> From: ginsb...@cisco.com At: 01/30/22 14:38:37 UTC-5:00
>
> To: rob...@raszuk.net, ketant.i...@gmail.com
> Cc: Albert Fu (BLOOMBERG/ 120 PARK ) , a...@cisco.com,
> draft-ietf-lsr-ospf-bfd-strict-m...@ietf.org, lsr@ietf.org
> Subject: RE: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD"
> - draft-ietf-lsr-ospf-bfd-strict-mode-04
>
>
>
> Robert –
>
>
>
> Here is what you said (emphasis added):
>
>
>
> 
>
> But the timer I am suggesting is not related to BFD operation, but to OSPF
> (and/or ISIS). It is not about BFD sessions being UP or DOWN. It is about 
> *allowing
> BFD for more testing (with various parameters (for example increasing test
> packet size in some discrete steps)* before OSPF is happy to bring the
> adj. up.
>
> 
>
>
>
> Point #1: If you want BFD to do more testing (such as MTU testing) then
> clearly you need extensions to BFD (such as
> https://datatracker.ietf.org/doc/draft-ietf-bfd-large-packets/ )
>
>
>
> Point #2: The existing timers (as Ketan points out are mentioned in
> Section 5) are applied today at the OSPF level precisely because OSPF does
> not currently have strict-mode operation. So in a flapping scenario you
> could see the following behavior:
>
>
>
> a)BFD goes down
>
> b)OSPF goes down in response to BFD
>
> c)OSPF comes back up
>
> d)Link is still unstable – so traffic is being dropped some of the time –
> but perhaps OSPF adjacency stays up 

Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - draft-ietf-lsr-ospf-bfd-strict-mode-04

[Albert Fu (BLOOMBERG/ 120 PARK)]

Hi Robert,

The BGP BFD hold-time mentioned in the BGP BFD strict mode draft has different 
meaning from the holdtime/delay/dampening that has been discussed in this forum 
thus far. 

The BGP BFD hold-time, as per the BGP BFD draft below, is user configurable, 
and is used to bring down the BGP session if BFD session is not established 
within default of 30s, when the negotiated "BGP HoldTimer" is 0. 


The OSPF hold-time/delay/dampening that we have been discussing so far is the 
delay from when BFD comes up to when OSPF will be allowed to come up. This, as 
Ketan mentioned, is outside the scope of this draft. 

In my testing with both Cisco and Juniper implementation, the OSPF 
hold-time/delay/dampening timers are quite arbitrary. You could have no delay 
(which means bring up OSPF asap), or have it configured on one side only. It 
serves as a sanity check that there's indeed a working BFD for a period of time 
before OSPF adjacency is allowed to progress.

Thanks

Albert

From: rob...@raszuk.net At: 01/31/22 09:59:48 UTC-5:00To:  Albert Fu 
(BLOOMBERG/ 120 PARK ) ,  ginsb...@cisco.com,  ketant.i...@gmail.com
Cc:  a...@cisco.com,  draft-ietf-lsr-ospf-bfd-strict-m...@ietf.org,  
lsr@ietf.org
Subject: Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - 
draft-ietf-lsr-ospf-bfd-strict-mode-04

Les & Ketan
 
Nowadays, it is also common to see the "break-in-middle" failures. we use BFD 
to detect this sort of failure within sub-second. And to dampen this sort of 
break-in-middle failures, we will need to use BFD holdtime/dampening. 


Another data point to the above and this discussion which Albert is co-author 
of. 

Ref: https://datatracker.ietf.org/doc/html/draft-ietf-idr-bgp-bfd-strict-mode

Please see the below paragraph which clearly says BGP BFD Hold time: 

   If the BFD session does not transition to the Up state, and the
   HoldTimer has been negotiated to a non-zero value, the BGP FSM will
   close the session appropriately.  If the HoldTimer has been
   negotiated to a zero value, the session should be closed after a time
   of X.  This time X is referred as "BGP BFD Hold time".  The proposed
   default BGP BFD Hold time value is 30 seconds.  The BGP BFD Hold time
   value is configurable.

To me it is clear that BGP BFD Hold time is on the client side and here affects 
BGP FSM. 

Thx,
Robert.


From: ginsb...@cisco.com At: 01/30/22 14:38:37 UTC-5:00To:  rob...@raszuk.net,  
ketant.i...@gmail.com
Cc:  Albert Fu (BLOOMBERG/ 120 PARK ) ,  a...@cisco.com,  
draft-ietf-lsr-ospf-bfd-strict-m...@ietf.org,  lsr@ietf.org
Subject: RE: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - 
draft-ietf-lsr-ospf-bfd-strict-mode-04

 

Robert – 
  
Here is what you said (emphasis added): 
  
 
But the timer I am suggesting is not related to BFD operation, but to OSPF 
(and/or ISIS). It is not about BFD sessions being UP or DOWN. It is about 
allowing BFD for more testing (with various parameters (for example increasing 
test packet size in some discrete steps) before OSPF is happy to bring the adj. 
up. 
 
  
Point #1: If you want BFD to do more testing (such as MTU testing) then clearly 
you need extensions to BFD (such as 
https://datatracker.ietf.org/doc/draft-ietf-bfd-large-packets/ ) 
  
Point #2: The existing timers (as Ketan points out are mentioned in Section 5) 
are applied today at the OSPF level precisely because OSPF does not currently 
have strict-mode operation. So in a flapping scenario you could see the 
following  behavior: 
  
a)BFD goes down 
b)OSPF goes down in response to BFD 
c)OSPF comes back up  
d)Link is still unstable – so traffic is being dropped some of the time – but 
perhaps OSPF adjacency stays up (i.e., OSPF hellos get through often enough to 
keep the OSPF adjacency up) 
  
So some implementations have chosen to insert a delay following “b”. This 
doesn’t guarantee stability, but hopefully makes it less likely. And because 
OSPF today does NOT wait for BFD to come up, the delay has to be implemented at 
the OSPF  level. 
  
Once you have strict mode support, the sequence becomes: 
  
a)BFD goes down 
b)OSPF goes down in response to BFD 
c)BFD comes back up 
d)OSPF comes back up 
  
Now, if the concern is that BFD comes back up while the link is still unstable, 
the way to address that is to put a delay either before BFD attempts to bring 
up a new session or a delay after achieving UP state before it signals UP to 
its  clients – such as OSPF. This is a better solution because all BFD clients 
benefit from this. Ad if the link is still unstable, it is more likely that the 
BFD session will go down during the delay period than it would be for OSPF 
because the BFD timers are  significantly more aggressive. 
(BTW, this behavior can be done w/o a BFD protocol extension – it is purely an 
implementation choice.) 
  
From a design perspective, dampening is always best done at the lowest layer 
possible. In most cases, interface layer dampening is best. 

Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - draft-ietf-lsr-ospf-bfd-strict-mode-04

[Les Ginsberg (ginsberg)]

Robert –

The paragraph you quote below has to do with BGP behavior in the event “BFD 
session does not transition to the Up state”.
There is no disagreement about what the protocol (BGP or OSPF) should do in 
this case. The point of strict-mode is to “wait-for-BFD”.

You, however, are trying to introduce some additional requirements. To this end 
you said:

“What I find missing in the draft is a mutually (between OSPF peers) timer 
fired after BFD session is up which in OSPF could hold on allowing BFD to do 
some more testing before declaring adj to be established. I think just bringing 
OSPF adj immediately after the BFD session is up is not a good thing.”

So apparently you want BFD to signal UP – but have the protocol do nothing 
until BFD completes some additional testing. What then was the point of BFD 
signaling UP to OSPF? And since you want the additional testing to be done by 
BFD, what new signal should BFD send to OSPF when this is done?
The point of BFD sending UP to its clients is to indicate that BFD thinks the 
link has been verified from the BFD perspective. I do not see the point of 
sending two such signals. If you think current BFD testing is inadequate please 
ask for extensions to BFD (in the BFD WG).

You also said:

“BFD is a great tool to tell you if the end to end path is UP or DOWN. It was 
not designed to give you any characteristics or metrics for the path quality.”

I agree. But if you are now proposing that protocol adjacencies should not come 
up until certain link quality metrics are met (e.g., link loss, delay) – you 
are moving into an area that is completely out of scope of this draft.
I won’t dig deeper into what could be a very lengthy discussion. If you really 
want to pursue this idea, I suggest you write a new draft.

   Les

From: Robert Raszuk 
Sent: Monday, January 31, 2022 6:59 AM
To: Albert Fu ; Les Ginsberg (ginsberg) 
; Ketan Talaulikar 
Cc: Acee Lindem (acee) ; 
draft-ietf-lsr-ospf-bfd-strict-m...@ietf.org; lsr 
Subject: Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - 
draft-ietf-lsr-ospf-bfd-strict-mode-04

Les & Ketan

Nowadays, it is also common to see the "break-in-middle" failures. we use BFD 
to detect this sort of failure within sub-second. And to dampen this sort of 
break-in-middle failures, we will need to use BFD holdtime/dampening.

Another data point to the above and this discussion which Albert is co-author 
of.

Ref: https://datatracker.ietf.org/doc/html/draft-ietf-idr-bgp-bfd-strict-mode

Please see the below paragraph which clearly says BGP BFD Hold time:

   If the BFD session does not transition to the Up state, and the
   HoldTimer has been negotiated to a non-zero value, the BGP FSM will
   close the session appropriately.  If the HoldTimer has been
   negotiated to a zero value, the session should be closed after a time
   of X.  This time X is referred as "BGP BFD Hold time".  The proposed
   default BGP BFD Hold time value is 30 seconds.  The BGP BFD Hold time
   value is configurable.

To me it is clear that BGP BFD Hold time is on the client side and here affects 
BGP FSM.

Thx,
Robert.







From: ginsb...@cisco.com At: 01/30/22 14:38:37 
UTC-5:00
To: rob...@raszuk.net, 
ketant.i...@gmail.com
Cc: Albert Fu (BLOOMBERG/ 120 PARK )  , 
a...@cisco.com, 
draft-ietf-lsr-ospf-bfd-strict-m...@ietf.org,
 lsr@ietf.org
Subject: RE: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - 
draft-ietf-lsr-ospf-bfd-strict-mode-04

Robert –

Here is what you said (emphasis added):


But the timer I am suggesting is not related to BFD operation, but to OSPF 
(and/or ISIS). It is not about BFD sessions being UP or DOWN. It is about 
allowing BFD for more testing (with various parameters (for example increasing 
test packet size in some discrete steps) before OSPF is happy to bring the adj. 
up.


Point #1: If you want BFD to do more testing (such as MTU testing) then clearly 
you need extensions to BFD (such as 
https://datatracker.ietf.org/doc/draft-ietf-bfd-large-packets/ )

Point #2: The existing timers (as Ketan points out are mentioned in Section 5) 
are applied today at the OSPF level precisely because OSPF does not currently 
have strict-mode operation. So in a flapping scenario you could see the 
following behavior:

a)BFD goes down
b)OSPF goes down in response to BFD
c)OSPF comes back up
d)Link is still unstable – so traffic is being dropped some of the time – but 
perhaps OSPF adjacency stays up (i.e., OSPF hellos get through often enough to 
keep the OSPF adjacency up)

So some implementations have chosen to insert a delay following “b”. This 
doesn’t guarantee stability, but hopefully makes it less likely. And because 
OSPF today does NOT wait for BFD to come up, the delay has to be implemented at 
the 

Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - draft-ietf-lsr-ospf-bfd-strict-mode-04

[Robert Raszuk]

Les,

BFD does not have a notion to come up in a hidden way, start testing
the link and only after some defined per client protocol or per interface
peer period signal to the client its "UP" state.

BFD holdtime as defined in RFC5880 is to keep BFD sessions and therefore
data probes down (for example to smooth in time control plane churn when
1000s of BFD peers are to all come up in the same time) so it is completely
unrelated to the discussion about BFD clients and link testing before
brining OSPF adj up.

If you want to extend BFD further you are welcome to take it to the BFD WG.
I am cc-ing Greg here as he is one of the best active BFD experts.

But if so this draft should wait till that happens. Alternatively as it has
been suggested we could choose to keep BFD simple and have such timer on
the client side. The behaviour on the client is trivial - Do your client
action (bring IGP adj. up or BGP session up or insert static to the RIB etc
...) if timer X elapsed from BFD session going UP and during that time
there was no transition to DOWN.

Regards,
R.




On Mon, Jan 31, 2022 at 6:31 PM Les Ginsberg (ginsberg) 
wrote:

> Albert –
>
>
>
> We are in full agreement.
>
>
>
> Delays in bringing BFD backup after a previous failure may well be
> warranted in the break-in-middle scenarios.
>
> I am not convinced this needs to be standardized – seems quite appropriate
> as an implementation choice. But if any discussion were to occur in RFCs, I
> think it should be in some BFD document.
>
>
>
> As this draft is focused on OSPF protocol extensions, I don’t think BFD
> dampening needs to be discussed. In any case it should not alter the
> interaction between BFD and protocols. If it takes longer for BFD to come
> up that just means the OSPF adjacency will not come up either – which is
> exactly the behavior that is desired.
>
>
>
> Les
>
>
>
>
>
> *From:* Albert Fu (BLOOMBERG/ 120 PARK) 
> *Sent:* Monday, January 31, 2022 6:50 AM
> *To:* Les Ginsberg (ginsberg) ; ketant.i...@gmail.com;
> rob...@raszuk.net
> *Cc:* Acee Lindem (acee) ;
> draft-ietf-lsr-ospf-bfd-strict-m...@ietf.org; lsr@ietf.org
> *Subject:* RE: [Lsr] Working Group Last Call for "OSPF Strict-Mode for
> BFD" - draft-ietf-lsr-ospf-bfd-strict-mode-04
>
>
>
> Hi Les,
>
>
>
> Your scenario below is indeed something we have encountered in our
> production network in the non-strict scenario, due to "flapping" links,
> where routing protocol could come up before BFD due to "break-in-middle"
> link issue (interface stayed up, so routing protocol remained active).
> Strict mode will address this issue.
>
>
>
> Another point to add is that we do have as a standard on our interfaces to
> safeguard against flapping link by configuring interface
> hold-time/carrier-delay. However, this is only useful in situations where
> the link physically goes down (and fast detection is automatic in most
> implementation).
>
>
>
> Nowadays, it is also common to see the "break-in-middle" failures. we use
> BFD to detect this sort of failure within sub-second. And to dampen this
> sort of break-in-middle failures, we will need to use BFD
> holdtime/dampening.
>
>
>
> Thanks
>
>
>
> Albert
>
>
>
>
>
>
>
> From: ginsb...@cisco.com At: 01/30/22 14:38:37 UTC-5:00
>
> To: rob...@raszuk.net, ketant.i...@gmail.com
> Cc: Albert Fu (BLOOMBERG/ 120 PARK ) , a...@cisco.com,
> draft-ietf-lsr-ospf-bfd-strict-m...@ietf.org, lsr@ietf.org
> Subject: RE: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD"
> - draft-ietf-lsr-ospf-bfd-strict-mode-04
>
>
>
> Robert –
>
>
>
> Here is what you said (emphasis added):
>
>
>
> 
>
> But the timer I am suggesting is not related to BFD operation, but to OSPF
> (and/or ISIS). It is not about BFD sessions being UP or DOWN. It is about 
> *allowing
> BFD for more testing (with various parameters (for example increasing test
> packet size in some discrete steps)* before OSPF is happy to bring the
> adj. up.
>
> 
>
>
>
> Point #1: If you want BFD to do more testing (such as MTU testing) then
> clearly you need extensions to BFD (such as
> https://datatracker.ietf.org/doc/draft-ietf-bfd-large-packets/ )
>
>
>
> Point #2: The existing timers (as Ketan points out are mentioned in
> Section 5) are applied today at the OSPF level precisely because OSPF does
> not currently have strict-mode operation. So in a flapping scenario you
> could see the following behavior:
>
>
>
> a)BFD goes down
>
> b)OSPF goes down in response to BFD
>
> c)OSPF comes back up
>
> d)Link is still unstable – so traffic is being dropped some of the time –
> but perhaps OSPF adjacency stays up (i.e., OSPF hellos get through often
> enough to keep the OSPF adjacency up)
>
>
>
> So some implementations have chosen to insert a delay following “b”. This
> doesn’t guarantee stability, but hopefully makes it less likely. And
> because OSPF today does NOT wait for BFD to come up, the delay has to be
> implemented at the OSPF level.
>
>
>
> Once you have strict mode support, the 

Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - draft-ietf-lsr-ospf-bfd-strict-mode-04

[Les Ginsberg (ginsberg)]

Albert –

We are in full agreement.

Delays in bringing BFD backup after a previous failure may well be warranted in 
the break-in-middle scenarios.
I am not convinced this needs to be standardized – seems quite appropriate as 
an implementation choice. But if any discussion were to occur in RFCs, I think 
it should be in some BFD document.

As this draft is focused on OSPF protocol extensions, I don’t think BFD 
dampening needs to be discussed. In any case it should not alter the 
interaction between BFD and protocols. If it takes longer for BFD to come up 
that just means the OSPF adjacency will not come up either – which is exactly 
the behavior that is desired.

Les


From: Albert Fu (BLOOMBERG/ 120 PARK) 
Sent: Monday, January 31, 2022 6:50 AM
To: Les Ginsberg (ginsberg) ; ketant.i...@gmail.com; 
rob...@raszuk.net
Cc: Acee Lindem (acee) ; 
draft-ietf-lsr-ospf-bfd-strict-m...@ietf.org; lsr@ietf.org
Subject: RE: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - 
draft-ietf-lsr-ospf-bfd-strict-mode-04

Hi Les,

Your scenario below is indeed something we have encountered in our production 
network in the non-strict scenario, due to "flapping" links, where routing 
protocol could come up before BFD due to "break-in-middle" link issue 
(interface stayed up, so routing protocol remained active). Strict mode will 
address this issue.

Another point to add is that we do have as a standard on our interfaces to 
safeguard against flapping link by configuring interface 
hold-time/carrier-delay. However, this is only useful in situations where the 
link physically goes down (and fast detection is automatic in most 
implementation).

Nowadays, it is also common to see the "break-in-middle" failures. we use BFD 
to detect this sort of failure within sub-second. And to dampen this sort of 
break-in-middle failures, we will need to use BFD holdtime/dampening.

Thanks

Albert



From: ginsb...@cisco.com At: 01/30/22 14:38:37 
UTC-5:00
To: rob...@raszuk.net, 
ketant.i...@gmail.com
Cc: Albert Fu (BLOOMBERG/ 120 PARK )  , 
a...@cisco.com, 
draft-ietf-lsr-ospf-bfd-strict-m...@ietf.org,
 lsr@ietf.org
Subject: RE: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - 
draft-ietf-lsr-ospf-bfd-strict-mode-04

Robert –

Here is what you said (emphasis added):


But the timer I am suggesting is not related to BFD operation, but to OSPF 
(and/or ISIS). It is not about BFD sessions being UP or DOWN. It is about 
allowing BFD for more testing (with various parameters (for example increasing 
test packet size in some discrete steps) before OSPF is happy to bring the adj. 
up.


Point #1: If you want BFD to do more testing (such as MTU testing) then clearly 
you need extensions to BFD (such as 
https://datatracker.ietf.org/doc/draft-ietf-bfd-large-packets/ )

Point #2: The existing timers (as Ketan points out are mentioned in Section 5) 
are applied today at the OSPF level precisely because OSPF does not currently 
have strict-mode operation. So in a flapping scenario you could see the 
following behavior:

a)BFD goes down
b)OSPF goes down in response to BFD
c)OSPF comes back up
d)Link is still unstable – so traffic is being dropped some of the time – but 
perhaps OSPF adjacency stays up (i.e., OSPF hellos get through often enough to 
keep the OSPF adjacency up)

So some implementations have chosen to insert a delay following “b”. This 
doesn’t guarantee stability, but hopefully makes it less likely. And because 
OSPF today does NOT wait for BFD to come up, the delay has to be implemented at 
the OSPF level.

Once you have strict mode support, the sequence becomes:

a)BFD goes down
b)OSPF goes down in response to BFD
c)BFD comes back up
d)OSPF comes back up

Now, if the concern is that BFD comes back up while the link is still unstable, 
the way to address that is to put a delay either before BFD attempts to bring 
up a new session or a delay after achieving UP state before it signals UP to 
its clients – such as OSPF. This is a better solution because all BFD clients 
benefit from this. Ad if the link is still unstable, it is more likely that the 
BFD session will go down during the delay period than it would be for OSPF 
because the BFD timers are significantly more aggressive.
(BTW, this behavior can be done w/o a BFD protocol extension – it is purely an 
implementation choice.)

From a design perspective, dampening is always best done at the lowest layer 
possible. In most cases, interface layer dampening is best. If that is not 
reliable for some reason, then move one layer up – not two layers up.

   Les


From: Robert Raszuk mailto:rob...@raszuk.net>>
Sent: Sunday, January 30, 2022 10:05 AM
To: Ketan Talaulikar mailto:ketant.i...@gmail.com>>
Cc: Les Ginsberg (ginsberg) 

Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - draft-ietf-lsr-ospf-bfd-strict-mode-04

[Robert Raszuk]

Les & Ketan


> Nowadays, it is also common to see the "break-in-middle" failures. we use
> BFD to detect this sort of failure within sub-second. And to dampen this
> sort of break-in-middle failures, we will need to use BFD
> holdtime/dampening.
>

Another data point to the above and this discussion which Albert is
co-author of.

Ref:
https://datatracker.ietf.org/doc/html/draft-ietf-idr-bgp-bfd-strict-mode

Please see the below paragraph which clearly says *BGP BFD Hold time*:

   If the BFD session does not transition to the Up state, and the
   HoldTimer has been negotiated to a non-zero value, the BGP FSM will
   close the session appropriately.  If the HoldTimer has been
   negotiated to a zero value, the session should be closed after a time
   of X.  This time X is referred as "BGP BFD Hold time".  The proposed
   default BGP BFD Hold time value is 30 seconds.  The BGP BFD Hold time
   value is configurable.

To me it is clear that BGP BFD Hold time is on the client side and here
affects BGP FSM.

Thx,
Robert.







From: ginsb...@cisco.com At: 01/30/22 14:38:37 UTC-5:00
> To: rob...@raszuk.net, ketant.i...@gmail.com
> Cc: Albert Fu (BLOOMBERG/ 120 PARK ) , a...@cisco.com,
> draft-ietf-lsr-ospf-bfd-strict-m...@ietf.org, lsr@ietf.org
> Subject: RE: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD"
> - draft-ietf-lsr-ospf-bfd-strict-mode-04
>
> Robert –
>
>
>
> Here is what you said (emphasis added):
>
>
>
> 
>
> But the timer I am suggesting is not related to BFD operation, but to OSPF
> (and/or ISIS). It is not about BFD sessions being UP or DOWN. It is about 
> *allowing
> BFD for more testing (with various parameters (for example increasing test
> packet size in some discrete steps)* before OSPF is happy to bring the
> adj. up.
>
> 
>
>
>
> Point #1: If you want BFD to do more testing (such as MTU testing) then
> clearly you need extensions to BFD (such as
> https://datatracker.ietf.org/doc/draft-ietf-bfd-large-packets/ )
>
>
>
> Point #2: The existing timers (as Ketan points out are mentioned in
> Section 5) are applied today at the OSPF level precisely because OSPF does
> not currently have strict-mode operation. So in a flapping scenario you
> could see the following behavior:
>
>
>
> a)BFD goes down
>
> b)OSPF goes down in response to BFD
>
> c)OSPF comes back up
>
> d)Link is still unstable – so traffic is being dropped some of the time –
> but perhaps OSPF adjacency stays up (i.e., OSPF hellos get through often
> enough to keep the OSPF adjacency up)
>
>
>
> So some implementations have chosen to insert a delay following “b”. This
> doesn’t guarantee stability, but hopefully makes it less likely. And
> because OSPF today does NOT wait for BFD to come up, the delay has to be
> implemented at the OSPF level.
>
>
>
> Once you have strict mode support, the sequence becomes:
>
>
>
> a)BFD goes down
>
> b)OSPF goes down in response to BFD
>
> c)BFD comes back up
>
> d)OSPF comes back up
>
>
>
> Now, if the concern is that BFD comes back up while the link is still
> unstable, the way to address that is to put a delay either before BFD
> attempts to bring up a new session or a delay after achieving UP state
> before it signals UP to its clients – such as OSPF. This is a better
> solution because all BFD clients benefit from this. Ad if the link is still
> unstable, it is more likely that the BFD session will go down during the
> delay period than it would be for OSPF because the BFD timers are
> significantly more aggressive.
>
> (BTW, this behavior can be done w/o a BFD protocol extension – it is
> purely an implementation choice.)
>
>
>
> From a design perspective, dampening is always best done at the lowest
> layer possible. In most cases, interface layer dampening is best. If that
> is not reliable for some reason, then move one layer up – not two layers up.
>
>
>
>Les
>
>
>
>
>
> *From:* Robert Raszuk 
> *Sent:* Sunday, January 30, 2022 10:05 AM
> *To:* Ketan Talaulikar 
> *Cc:* Les Ginsberg (ginsberg) ; Acee Lindem (acee) <
> a...@cisco.com>; draft-ietf-lsr-ospf-bfd-strict-m...@ietf.org; Albert Fu <
> af...@bloomberg.net>; lsr 
> *Subject:* Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for
> BFD" - draft-ietf-lsr-ospf-bfd-strict-mode-04
>
>
>
> Hi Ketan,
>
>
>
> I would like to point out that the draft discusses the BFD "dampening" or
> "hold-down" mechanism in Sec 5. We are aware of BFD implementations that
> include such mechanisms in a protocol-agnostic manner.
>
>
>
> BFD dampening or hold-time are completely orthogonal to my point. Both
> have nothing to do with it.
>
>
>
> Those timers only fire when BFD goes down. In my example BFD does not go
> down. But we want to bring up the client adj. only after X ms/sec/min etc
> ...of normal BFD operation if no failure is detected during that timer.
>
>
>
> This draft indicates that OSPF adjacency will "advance" in the neighbor
> FSM only after BFD reports UP.
>
>
>
> And that is exactly too soon. In fact if you do 

Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - draft-ietf-lsr-ospf-bfd-strict-mode-04

[Albert Fu (BLOOMBERG/ 120 PARK)]

Hi Les,

Your scenario below is indeed something we have encountered in our production 
network in the non-strict scenario, due to "flapping" links, where routing 
protocol could come up before BFD due to "break-in-middle" link issue 
(interface stayed up, so routing protocol remained active). Strict mode will 
address this issue. 

Another point to add is that we do have as a standard on our interfaces to 
safeguard against flapping link by configuring interface 
hold-time/carrier-delay. However, this is only useful in situations where the 
link physically goes down (and fast detection is automatic in most 
implementation). 

Nowadays, it is also common to see the "break-in-middle" failures. we use BFD 
to detect this sort of failure within sub-second. And to dampen this sort of 
break-in-middle failures, we will need to use BFD holdtime/dampening. 

Thanks

Albert


From: ginsb...@cisco.com At: 01/30/22 14:38:37 UTC-5:00To:  rob...@raszuk.net,  
ketant.i...@gmail.com
Cc:  Albert Fu (BLOOMBERG/ 120 PARK ) ,  a...@cisco.com,  
draft-ietf-lsr-ospf-bfd-strict-m...@ietf.org,  lsr@ietf.org
Subject: RE: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - 
draft-ietf-lsr-ospf-bfd-strict-mode-04

 

Robert – 
  
Here is what you said (emphasis added): 
  
 
But the timer I am suggesting is not related to BFD operation, but to OSPF 
(and/or ISIS). It is not about BFD sessions being UP or DOWN. It is about 
allowing BFD for more testing (with various parameters (for example increasing 
test packet size in some discrete steps) before OSPF is happy to bring the adj. 
up. 
 
  
Point #1: If you want BFD to do more testing (such as MTU testing) then clearly 
you need extensions to BFD (such as 
https://datatracker.ietf.org/doc/draft-ietf-bfd-large-packets/ ) 
  
Point #2: The existing timers (as Ketan points out are mentioned in Section 5) 
are applied today at the OSPF level precisely because OSPF does not currently 
have strict-mode operation. So in a flapping scenario you could see the 
following  behavior: 
  
a)BFD goes down 
b)OSPF goes down in response to BFD 
c)OSPF comes back up  
d)Link is still unstable – so traffic is being dropped some of the time – but 
perhaps OSPF adjacency stays up (i.e., OSPF hellos get through often enough to 
keep the OSPF adjacency up) 
  
So some implementations have chosen to insert a delay following “b”. This 
doesn’t guarantee stability, but hopefully makes it less likely. And because 
OSPF today does NOT wait for BFD to come up, the delay has to be implemented at 
the OSPF  level. 
  
Once you have strict mode support, the sequence becomes: 
  
a)BFD goes down 
b)OSPF goes down in response to BFD 
c)BFD comes back up 
d)OSPF comes back up 
  
Now, if the concern is that BFD comes back up while the link is still unstable, 
the way to address that is to put a delay either before BFD attempts to bring 
up a new session or a delay after achieving UP state before it signals UP to 
its  clients – such as OSPF. This is a better solution because all BFD clients 
benefit from this. Ad if the link is still unstable, it is more likely that the 
BFD session will go down during the delay period than it would be for OSPF 
because the BFD timers are  significantly more aggressive. 
(BTW, this behavior can be done w/o a BFD protocol extension – it is purely an 
implementation choice.) 
  
From a design perspective, dampening is always best done at the lowest layer 
possible. In most cases, interface layer dampening is best. If that is not 
reliable for some reason, then move one layer up – not two layers up. 
  
   Les 
  
  

From: Robert Raszuk  
Sent: Sunday, January 30, 2022 10:05 AM
To: Ketan Talaulikar 
Cc: Les Ginsberg (ginsberg) ; Acee Lindem (acee) 
; draft-ietf-lsr-ospf-bfd-strict-m...@ietf.org; Albert Fu 
; lsr 
Subject: Re: [Lsr] Working Group Last Call for "OSPF Strict-Mode for BFD" - 
draft-ietf-lsr-ospf-bfd-strict-mode-04 
  

Hi Ketan, 

  


I would like to point out that the draft discusses the BFD "dampening" or 
"hold-down" mechanism in Sec 5. We are aware of BFD implementations that 
include such mechanisms in a protocol-agnostic manner. 
 

  

BFD dampening or hold-time are completely orthogonal to my point. Both have 
nothing to do with it.  

  

Those timers only fire when BFD goes down. In my example BFD does not go down. 
But we want to bring up the client adj. only after X ms/sec/min etc ...of 
normal BFD operation if no failure is detected during that timer. 

  

This draft indicates that OSPF adjacency will "advance" in the neighbor FSM 
only after BFD reports UP.  
 

  

And that is exactly too soon. In fact if you do that today without waiting some 
time (if you retire the current OSPF timer) you will not help at all in the 
case you are trying to address.  

  

Reason being that perhaps 200 ms after BFD UP it will go down, but OSPF adj. 
will get already established. It is really pretty simple.  

  

Thx, 

Robert. 

  

PS. And yes I