Re: [netmod] Unknown bits - backwards compatibility

[Kent Watsen]

H Jeff,

> I hadn’t realized that your intent was to skip directly to WGLC, unless this 
> was a typo.  Most WG process I deal with goes through at least a thin 
> adoption stage even if the intent is to move forward swiftly to last call.

I meant “adoption”.  No process skipping here!


>>  Specifically, I’m wondering if it makes sense to add a new section to 
>> provide guidance to implementors?   I’m unsure myself, as the concerns 
>> raised seem to be addressed by YANG Library, in that clients should use it 
>> to detect if the server is using a newer revision of modules than the client 
>> groks.
> 
> As Jason notes in his reply, I’m unclear what YANG library may do to help 
> this situation.

As Martin points out, a revision of a module using this typedef is backwards 
compatible (both the old and new modules define the same “unknown-flags” leaf 
with the same meaning, and adding new bits to the “flags” leaf is allowed per 
RFC 7950, Section 11.  The concern isn’t that the module revision is NBC, but 
that the data changed and that, a client coded to look-for/act-on the presence 
of an unknown bit would break, which could be construed as NBC at the *data* 
level.  My point with regards to YL is that this is nothing new.  New revisions 
of modules can always come out with new nodes and clients always need to update 
themselves to understand the module versions used by the server.

In this case, the debate is particularly shallow.  Firstly, I think that it is 
bad form for a client to be coded to look-for/act-on the “unknown bits”.  So 
the "NBC at the data level” case shouldn’t occur in practice.  Next, as my edit 
below alludes, when a bit goes from being unknown to known, a new revision of 
the module will be defined and implemented by the server, meaning the server 
will start sending the new bit in the, e.g., “flags” leaf.  That the server 
stopped sending the bit in the “unknown-flags” leaf is secondary to that 
started sending it in the “flags” leaf.   A client that “breaks" because an 
unknown-bit isn’t sent will more surely break  because a new known bit is sent. 
 Again, one purpose of YL is for clients to ensure they’re up to date wrt to 
the servers it connects to.


> I’m certainly fine adding an additional section to the draft discussing that 
> unknown bits may “disappear” during maintenance as previously unknown bits 
> become defined.

Something that captures some of what I wrote above?  


>> One new comment: the "unknown-bits” typedef's description statement seems 
>> unclear in terms of the expected reconciliation process.
>> 
>> Current:
>>   "Typedef describing 64 bits worth of unknown bits.  This can be
>>used to model operational state that would normally be modeled
>>using the YANG 'bits' type, but no registered bit has been
>>created.”;
>> 
>> Proposed:
>> 
>>   "Typedef describing 64 bits worth of unknown bits.  This can be
>>used to model operational state that would normally be modeled
>>using the YANG 'bits' type, but no registered bit has been
>>defined in the YANG module implemented by the server.”;
>> 
> 
> I’m fine making this change.

I refrained from expanding the description statement even more, but leave it to 
you to consider.  I was thinking to also add something like:

This typedef is intended to be used to model a “catch all” leaf for when, e.g., 
over-the-wire protocol data contains bits not representable by the module 
implemented by the server.  The “unknown-bits” data should be considered 
short-lived, as the bits will be known as soon as the server updates to a 
module revision defining them.   Implementations should not look for or act on 
the presence of unknown bits.


And then, for implementations that disregard that advice, they can always 
update themselves:

OLD

if unknown-flags.bit-3:
foo

NEW

if unknown-flags.bit-3 or known-flags.my-foo-bit:
foo

I don’t see the reason to force servers to carry-forwards (forever) the 
reporting of previously unknown bits.


> — Jeff

K.


___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Jeff Haas]

Jason,

On Jun 28, 2023, at 5:51 PM, Jason Sterne (Nokia)  
wrote:
> 
> 
> I’m not sure how the concerns about non-backwards-compatibility are addressed 
> by YANG library. Both Rob and I were proposing that one potential solution is 
> to always send all the bits in the 2nd leaf (even after they become “known 
> bits” in the first leaf).

As previously discussed in the thread, while this is a possible mechanism, it’s 
not the intended use case.  Certainly the typedef proposed here could be used 
for such a case where you’re always defining the bits, but in my opinion it’d 
be better in such cases to simply use type “binary” to model such things.

A reminder of the use case I’m trying to cover:
Protocols will typically move across their lifetime from mostly “known” bits in 
their bit vectors/flag fields during their definition with some bits left over 
as reserved.  Over a period of maintenance, previously unknown bits may be 
used.  Implementations of a YANG module covering an older protocol definition 
using bits or identities have no way to operationally represent that version of 
the module’s “unknown” state.

The intent is to cover two cases: Transient (from the perspective of one module 
version to the next) missing definitions of bits that will become known. 
Protocol violations where unknown bits are sent and would not be expected even 
in updated protocol versions.

For the first case, only a few bits would typically be expected to show up from 
one version of the module to the next.  Having an “unknown” pop up in the 
unknown leaf should be an easy, “Surprise!” But if it’s buried in the noise of 
the normal, it doesn’t stand out.

Distinguishing the first from the second case is impossible from the 
perspective of a given version of the module.  All it knows is that the bit is 
unknown.  The operator would have to know after study that a previously unknown 
bit is getting signaled and should not be after researching protocol changes.  
But at least they get the operational prod to investigate the WTF moment.

After such moments, it’s not atypical to break out tcpdump. In such cases, 
access to raw contents can become important.  In current modeling practices, 
I’m of mixed opinion about always putting raw contents of packets in management 
data; it adds significant noise.  I’d personally prefer the modeling 
methodologies and tooling evolve to a mode where such deep debug data were 
either circumstantially present (it’s not noise when you ask for it) or in a 
distinct debug sub-tree.

— Jeff
___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Jeff Haas]

Kent,


> On Jun 28, 2023, at 4:25 PM, Kent Watsen  wrote:
> I’ve been hoping you would reply to some of the comments here before kicking 
> off the WGLC.

I hadn’t realized that your intent was to skip directly to WGLC, unless this 
was a typo.  Most WG process I deal with goes through at least a thin adoption 
stage even if the intent is to move forward swiftly to last call.

>  Specifically, I’m wondering if it makes sense to add a new section to 
> provide guidance to implementors?   I’m unsure myself, as the concerns raised 
> seem to be addressed by YANG Library, in that clients should use it to detect 
> if the server is using a newer revision of modules than the client groks.

As Jason notes in his reply, I’m unclear what YANG library may do to help this 
situation.

I’m certainly fine adding an additional section to the draft discussing that 
unknown bits may “disappear” during maintenance as previously unknown bits 
become defined.

> 
> One new comment: the "unknown-bits” typedef's description statement seems 
> unclear in terms of the expected reconciliation process.
> 
> Current:
>   "Typedef describing 64 bits worth of unknown bits.  This can be
>used to model operational state that would normally be modeled
>using the YANG 'bits' type, but no registered bit has been
>created.”;
> 
> Proposed:
> 
>   "Typedef describing 64 bits worth of unknown bits.  This can be
>used to model operational state that would normally be modeled
>using the YANG 'bits' type, but no registered bit has been
>defined in the YANG module implemented by the server.”;
> 

I’m fine making this change.

— Jeff


> Thoughts?
> 
> K.
> 
> 
> 
>> On May 23, 2023, at 11:59 AM, Rob Wilton (rwilton) 
>>  wrote:
>> 
>> I was looking at this draft again, and since I had read the -02 version of 
>> the draft I thought that I would send the comments here.
>> 
>> My no hats view here (looking at the latest draft) is:
>> documenting something here is probably helpful because this is an issue that 
>> folks are experiencing and providing written guidance on how to handle it 
>> helpful to the YANG modelling community.
>> having such a document update RFC 8407 would probably be helpful (to help 
>> YANG authors find it),
>> but I’m concerned that the proposed solution is not backwards compatible at 
>> the instance data level.  E.g., an old client talking to a new server would 
>> still hit a problem if the server changed from sending an unknown "bit-" 
>> to a known bit (defined in a backards-compatible module update), and if the 
>> old client doesn't know/understand the now known bit (because they are still 
>> coded against the older module version), then the client may break.  This 
>> makes me think that there is going to be some subtleties around when it is 
>> safe to not send an unknown bit, hence my suggestion was to call it just 
>> “bits” (which is already in -02) and *always sending* this leaf may be a 
>> more backwards compatible choice.  It isn’t clear to me, for the “all bits” 
>> leaf, whether sending this as a raw uint8 – uint64, or binary, and doing the 
>> decode on the client side (if required) might be a better choice than 
>> sending them as a generic bits type.
>> 
>> Regards,
>> Rob
>>  
>> From: netmod  On Behalf Of Italo Busi
>> Sent: 14 April 2023 09:05
>> To: Jason Sterne (Nokia) ; Jeffrey Haas 
>> ; netmod@ietf.org
>> Subject: Re: [netmod] Unknown bits - backwards compatibility
>>  
>> Jason, Jeffrey,
>>  
>> If the need is to report the value of a received protocol field for 
>> debugging/troubleshooting purposes, I am wondering why not using a binary 
>> type instead of a bits type or, better, a YANG leaf of bits type for the 
>> known bits and another YANG leaf of binary type for all known/unknown bits
>>  
>> Trailing zeros can be added when the protocol field is not an integer number 
>> of octets
>>  
>> In this way there will be no backward compatibility issues when unknown bits 
>> are assigned and becomes known
>>  
>> My 2 cents
>>  
>> Italo
>>  
>> From: Jason Sterne (Nokia)  
>> Sent: mercoledì 12 aprile 2023 23:26
>> To: Jason Sterne (Nokia) ; Jeffrey Haas 
>> ; netmod@ietf.org
>> Subject: Re: [netmod] Unknown bits - backwards compatibility
>>  
>> It just occurred to me that to avoid the NBC change we could also consider 
>> just calling this new typedef “raw-bits” and always outputting all the bits 
>> in the second leaf (whether they are known or not)?  I susp

Re: [netmod] Unknown bits - backwards compatibility

[Jason Sterne (Nokia)]

I’m not sure how the concerns about non-backwards-compatibility are addressed 
by YANG library. Both Rob and I were proposing that one potential solution is 
to always send all the bits in the 2nd leaf (even after they become “known 
bits” in the first leaf).

From: Kent Watsen 
Sent: Wednesday, June 28, 2023 4:25 PM
To: Jeffrey Haas 
Cc: Rob Wilton (rwilton) ; Italo Busi 
; Jason Sterne (Nokia) 
; netmod@ietf.org
Subject: Re: [netmod] Unknown bits - backwards compatibility


CAUTION: This is an external email. Please be very careful when clicking links 
or opening attachments. See the URL nok.it/ext for additional information.


Hi Jeff,

I’ve been hoping you would reply to some of the comments here before kicking 
off the WGLC.  Specifically, I’m wondering if it makes sense to add a new 
section to provide guidance to implementors?   I’m unsure myself, as the 
concerns raised seem to be addressed by YANG Library, in that clients should 
use it to detect if the server is using a newer revision of modules than the 
client groks.

One new comment: the "unknown-bits” typedef's description statement seems 
unclear in terms of the expected reconciliation process.


Current:



  "Typedef describing 64 bits worth of unknown bits.  This can be

   used to model operational state that would normally be modeled

   using the YANG 'bits' type, but no registered bit has been

   created.”;



Proposed:



  "Typedef describing 64 bits worth of unknown bits.  This can be

   used to model operational state that would normally be modeled

   using the YANG 'bits' type, but no registered bit has been

   defined in the YANG module implemented by the server.”;



Thoughts?


K.




On May 23, 2023, at 11:59 AM, Rob Wilton (rwilton) 
mailto:rwilton=40cisco@dmarc.ietf.org>> 
wrote:

I was looking at this draft again, and since I had read the -02 version of the 
draft I thought that I would send the comments here.


My no hats view here (looking at the latest draft) is:

  *   documenting something here is probably helpful because this is an issue 
that folks are experiencing and providing written guidance on how to handle it 
helpful to the YANG modelling community.
  *   having such a document update RFC 8407 would probably be helpful (to help 
YANG authors find it),
  *   but I’m concerned that the proposed solution is not backwards compatible 
at the instance data level.  E.g., an old client talking to a new server would 
still hit a problem if the server changed from sending an unknown "bit-" to 
a known bit (defined in a backards-compatible module update), and if the old 
client doesn't know/understand the now known bit (because they are still coded 
against the older module version), then the client may break.  This makes me 
think that there is going to be some subtleties around when it is safe to not 
send an unknown bit, hence my suggestion was to call it just “bits” (which is 
already in -02) and *always sending* this leaf may be a more backwards 
compatible choice.  It isn’t clear to me, for the “all bits” leaf, whether 
sending this as a raw uint8 – uint64, or binary, and doing the decode on the 
client side (if required) might be a better choice than sending them as a 
generic bits type.


Regards,
Rob

From: netmod mailto:netmod-boun...@ietf.org>> On 
Behalf Of Italo Busi
Sent: 14 April 2023 09:05
To: Jason Sterne (Nokia) 
mailto:jason.ste...@nokia.com>>; Jeffrey Haas 
mailto:jh...@pfrc.org>>; netmod@ietf.org<mailto:netmod@ietf.org>
Subject: Re: [netmod] Unknown bits - backwards compatibility

Jason, Jeffrey,

If the need is to report the value of a received protocol field for 
debugging/troubleshooting purposes, I am wondering why not using a binary type 
instead of a bits type or, better, a YANG leaf of bits type for the known bits 
and another YANG leaf of binary type for all known/unknown bits

Trailing zeros can be added when the protocol field is not an integer number of 
octets

In this way there will be no backward compatibility issues when unknown bits 
are assigned and becomes known

My 2 cents

Italo

From: Jason Sterne (Nokia) 
mailto:jason.ste...@nokia.com>>
Sent: mercoledì 12 aprile 2023 23:26
To: Jason Sterne (Nokia) 
mailto:jason.ste...@nokia.com>>; Jeffrey Haas 
mailto:jh...@pfrc.org>>; netmod@ietf.org<mailto:netmod@ietf.org>
Subject: Re: [netmod] Unknown bits - backwards compatibility

It just occurred to me that to avoid the NBC change we could also consider just 
calling this new typedef “raw-bits” and always outputting all the bits in the 
second leaf (whether they are known or not)?  I suspect this was already 
considered though…

Jason

From: netmod mailto:netmod-boun...@ietf.org>> On 
Behalf Of Jason Sterne (Nokia)
Sent: Wednesday, April 12, 2023 5:24 PM
To: Jeffrey Haas mailto:jh...@pfrc.org>>; 
netmod@ietf.org<mailto:netmod@ietf.org>
Subject: [net

Re: [netmod] Unknown bits - backwards compatibility

[Kent Watsen]

Hi Jeff,

I’ve been hoping you would reply to some of the comments here before kicking 
off the WGLC.  Specifically, I’m wondering if it makes sense to add a new 
section to provide guidance to implementors?   I’m unsure myself, as the 
concerns raised seem to be addressed by YANG Library, in that clients should 
use it to detect if the server is using a newer revision of modules than the 
client groks.

One new comment: the "unknown-bits” typedef's description statement seems 
unclear in terms of the expected reconciliation process.

Current:

  "Typedef describing 64 bits worth of unknown bits.  This can be
   used to model operational state that would normally be modeled
   using the YANG 'bits' type, but no registered bit has been
   created.”;

Proposed:

  "Typedef describing 64 bits worth of unknown bits.  This can be
   used to model operational state that would normally be modeled
   using the YANG 'bits' type, but no registered bit has been
   defined in the YANG module implemented by the server.”;

Thoughts?

K.



> On May 23, 2023, at 11:59 AM, Rob Wilton (rwilton) 
>  wrote:
> 
> I was looking at this draft again, and since I had read the -02 version of 
> the draft I thought that I would send the comments here.
> 
> My no hats view here (looking at the latest draft) is:
> documenting something here is probably helpful because this is an issue that 
> folks are experiencing and providing written guidance on how to handle it 
> helpful to the YANG modelling community.
> having such a document update RFC 8407 would probably be helpful (to help 
> YANG authors find it),
> but I’m concerned that the proposed solution is not backwards compatible at 
> the instance data level.  E.g., an old client talking to a new server would 
> still hit a problem if the server changed from sending an unknown "bit-" 
> to a known bit (defined in a backards-compatible module update), and if the 
> old client doesn't know/understand the now known bit (because they are still 
> coded against the older module version), then the client may break.  This 
> makes me think that there is going to be some subtleties around when it is 
> safe to not send an unknown bit, hence my suggestion was to call it just 
> “bits” (which is already in -02) and *always sending* this leaf may be a more 
> backwards compatible choice.  It isn’t clear to me, for the “all bits” leaf, 
> whether sending this as a raw uint8 – uint64, or binary, and doing the decode 
> on the client side (if required) might be a better choice than sending them 
> as a generic bits type.
> 
> Regards,
> Rob
>  
> From: netmod mailto:netmod-boun...@ietf.org>> On 
> Behalf Of Italo Busi
> Sent: 14 April 2023 09:05
> To: Jason Sterne (Nokia)  <mailto:jason.ste...@nokia.com>>; Jeffrey Haas  <mailto:jh...@pfrc.org>>; netmod@ietf.org <mailto:netmod@ietf.org>
> Subject: Re: [netmod] Unknown bits - backwards compatibility
>  
> Jason, Jeffrey,
>  
> If the need is to report the value of a received protocol field for 
> debugging/troubleshooting purposes, I am wondering why not using a binary 
> type instead of a bits type or, better, a YANG leaf of bits type for the 
> known bits and another YANG leaf of binary type for all known/unknown bits
>  
> Trailing zeros can be added when the protocol field is not an integer number 
> of octets
>  
> In this way there will be no backward compatibility issues when unknown bits 
> are assigned and becomes known
>  
> My 2 cents
>  
> Italo
>  
> From: Jason Sterne (Nokia)  <mailto:jason.ste...@nokia.com>> 
> Sent: mercoledì 12 aprile 2023 23:26
> To: Jason Sterne (Nokia)  <mailto:jason.ste...@nokia.com>>; Jeffrey Haas  <mailto:jh...@pfrc.org>>; netmod@ietf.org <mailto:netmod@ietf.org>
> Subject: Re: [netmod] Unknown bits - backwards compatibility
>  
> It just occurred to me that to avoid the NBC change we could also consider 
> just calling this new typedef “raw-bits” and always outputting all the bits 
> in the second leaf (whether they are known or not)?  I suspect this was 
> already considered though…
>  
> Jason
>  
> From: netmod mailto:netmod-boun...@ietf.org>> On 
> Behalf Of Jason Sterne (Nokia)
> Sent: Wednesday, April 12, 2023 5:24 PM
> To: Jeffrey Haas mailto:jh...@pfrc.org>>; netmod@ietf.org 
> <mailto:netmod@ietf.org>
> Subject: [netmod] Unknown bits - backwards compatibility
>  
> Hi Jeff,
>  
> One topic that came up during the IETF 116 NETMOD meeting was backwards 
> compatibility.
>  
> From what I understand, a leaf (e.g. unknown-flags) that uses the 
> unknown-bits typedef would never change its definition in YANG. It would 
> always be d

Re: [netmod] Unknown bits - backwards compatibility

[Rob Wilton (rwilton)]

I was looking at this draft again, and since I had read the -02 version of the 
draft I thought that I would send the comments here.

My no hats view here (looking at the latest draft) is:

  *   documenting something here is probably helpful because this is an issue 
that folks are experiencing and providing written guidance on how to handle it 
helpful to the YANG modelling community.
  *   having such a document update RFC 8407 would probably be helpful (to help 
YANG authors find it),
  *   but I'm concerned that the proposed solution is not backwards compatible 
at the instance data level.  E.g., an old client talking to a new server would 
still hit a problem if the server changed from sending an unknown "bit-" to 
a known bit (defined in a backards-compatible module update), and if the old 
client doesn't know/understand the now known bit (because they are still coded 
against the older module version), then the client may break.  This makes me 
think that there is going to be some subtleties around when it is safe to not 
send an unknown bit, hence my suggestion was to call it just "bits" (which is 
already in -02) and *always sending* this leaf may be a more backwards 
compatible choice.  It isn't clear to me, for the "all bits" leaf, whether 
sending this as a raw uint8 - uint64, or binary, and doing the decode on the 
client side (if required) might be a better choice than sending them as a 
generic bits type.

Regards,
Rob

From: netmod  On Behalf Of Italo Busi
Sent: 14 April 2023 09:05
To: Jason Sterne (Nokia) ; Jeffrey Haas 
; netmod@ietf.org
Subject: Re: [netmod] Unknown bits - backwards compatibility

Jason, Jeffrey,

If the need is to report the value of a received protocol field for 
debugging/troubleshooting purposes, I am wondering why not using a binary type 
instead of a bits type or, better, a YANG leaf of bits type for the known bits 
and another YANG leaf of binary type for all known/unknown bits

Trailing zeros can be added when the protocol field is not an integer number of 
octets

In this way there will be no backward compatibility issues when unknown bits 
are assigned and becomes known

My 2 cents

Italo

From: Jason Sterne (Nokia) 
mailto:jason.ste...@nokia.com>>
Sent: mercoledì 12 aprile 2023 23:26
To: Jason Sterne (Nokia) 
mailto:jason.ste...@nokia.com>>; Jeffrey Haas 
mailto:jh...@pfrc.org>>; netmod@ietf.org<mailto:netmod@ietf.org>
Subject: Re: [netmod] Unknown bits - backwards compatibility

It just occurred to me that to avoid the NBC change we could also consider just 
calling this new typedef "raw-bits" and always outputting all the bits in the 
second leaf (whether they are known or not)?  I suspect this was already 
considered though...

Jason

From: netmod mailto:netmod-boun...@ietf.org>> On 
Behalf Of Jason Sterne (Nokia)
Sent: Wednesday, April 12, 2023 5:24 PM
To: Jeffrey Haas mailto:jh...@pfrc.org>>; 
netmod@ietf.org<mailto:netmod@ietf.org>
Subject: [netmod] Unknown bits - backwards compatibility

Hi Jeff,

One topic that came up during the IETF 116 NETMOD meeting was backwards 
compatibility.

>From what I understand, a leaf (e.g. unknown-flags) that uses the unknown-bits 
>typedef would never change its definition in YANG. It would always be defined 
>as unknown-bits with all 64 bit definitions even as more and more bits become 
>"known".  *But* the system would suddenly stop reporting bit-0, then bit-1 in 
>that unknown-flags leaf as those bits become known.

Strictly speaking, that should probably be considered an NBC change although it 
is a bit of a grey zone - the NBC rules for state are fuzzy (theoretically they 
are defined by RFC7950 but it is clear those rules were focused on config, and 
the same goes for all our new versioning work). But I could imagine an 
implementation that was previously seeing bit-0 returned and suddenly stops 
seeing that bit-0 returned (which could cause different 
interpretation/behavior). So in some ways the semantics of the unknown-flags 
leaf has changed. It may be better to just flag this as an NBC change so a user 
would be drawn to look at it, and make a decision that they do or don't have to 
update their handling/use of it?

The known flags leaf would only go through backwards compatible changes though 
(since we'd always be additive in adding bits) - assuming bit positions don't 
change in the protocol.

It would probably help to show an example of what a YANG module looks like for 
step 1 and then step 2 (an unknown becomes known), and also what is returned in 
NETCONF in step 1 and step 2. You partially have that in section 3.2 although 
the YANG model isn't shown and it might be good to explicitly mention that 
 isn't returned (note it may also be valid to return 
).

Jason


From: netmod mailto:netmod-boun...@ietf.org>> On 
Behalf Of Jeffrey Haas
Sent: Monday, April 10, 2023 2:48 PM
To: netmod@ietf.

Re: [netmod] Unknown bits - backwards compatibility

[Jeffrey Haas]

However, the use case is covered by two points:
1. Don't emit the "unknown" node when you know everything.
2. Encodings that make you "hunt for the bit" are a PITA.  If bit-31 is the 
only new one in a 32-bit vector, you have to know that's the thing to look for 
if it's in any format other than bits. xx-yy-zz-q1 - we care about the 1 here 
and have to "know" to ignore xx-yy-zz-q.

Never mind that my actual example comes from an RFC where the bit vector is a 
nybble and hex-string requires a full octet of representation.

-- Jeff


> On Apr 17, 2023, at 9:40 AM, Jason Sterne (Nokia)  
> wrote:
> 
> That could be one approach, but note that the final step (your last sentence) 
> would still be NBC.
>  
> From: Rob Wilton (rwilton)  
> Sent: Monday, April 17, 2023 9:36 AM
> To: Jason Sterne (Nokia) ; Italo Busi 
> ; Jeffrey Haas 
> Cc: netmod@ietf.org
> Subject: RE: [netmod] Unknown bits - backwards compatibility
>  
>  
> CAUTION: This is an external email. Please be very careful when clicking 
> links or opening attachments. See the URL nok.it/ext for additional 
> information.
>  
> 
> Hi Jason,
>  
> Yes, I was thinking that you would always return the raw value and the bits 
> encoded value, but perhaps a server could optionally elide the raw value when 
> it knows that it can be fully represented by the bits value.
>  
> Regards,
> Rob
>  
>  
>  
> From: Jason Sterne (Nokia)  <mailto:jason.ste...@nokia.com>> 
> Sent: 17 April 2023 14:05
> To: Rob Wilton (rwilton) mailto:rwil...@cisco.com>>; 
> Italo Busi mailto:italo.b...@huawei.com>>; Jeffrey 
> Haas mailto:jh...@pfrc.org>>
> Cc: netmod@ietf.org <mailto:netmod@ietf.org>
> Subject: RE: [netmod] Unknown bits - backwards compatibility
>  
> Rob & Italo – are you proposing that the “raw-bits” are all always returned 
> (whether they are known or not)? 
> Jason
>  
> From: netmod mailto:netmod-boun...@ietf.org>> On 
> Behalf Of Rob Wilton (rwilton)
> Sent: Monday, April 17, 2023 7:43 AM
> To: Italo Busi  <mailto:Italo.Busi=40huawei@dmarc.ietf.org>>; Jeffrey Haas 
> mailto:jh...@pfrc.org>>
> Cc: netmod@ietf.org <mailto:netmod@ietf.org>
> Subject: Re: [netmod] Unknown bits - backwards compatibility
>  
>  
> CAUTION: This is an external email. Please be very careful when clicking 
> links or opening attachments. See the URL nok.it/ext for additional 
> information.
>  
> 
> Italo’s suggest was also how I was thinking of this.
>  
> We could define a convention for how the “raw” leaf should be named relative 
> to the bits decoded leaf, and also what type the “raw” leaf should use.  
> E.g., in the case where the length of the bits field is known (e.g., it is 
> defined/constrained by a protocol spec), then it could just use a uint8 – 
> through uint64 as appropriate, otherwise I would have thought that binary 
> would be appropriate underlying type.
>  
> As new bits get defined then the bits type would get updated, but the “raw” 
> leaf wouldn’t need to change.
>  
> Hence a draft could:
> Define a convention for naming the “raw value” leaves to be used in 
> conjunction with the decodable bits types leaves.
> Define typedefs that wrap (uint8 – uint64, and binary) to indicate the 
> semantics as representing undecoded bits.
> Define a convention for decoding/displaying the new types (perhaps in 
> conjunction with the decoded bits type leaf).
>  
> Regards,
> Rob
>  
> // As a contributor.
>  
>  
> From: netmod mailto:netmod-boun...@ietf.org>> On 
> Behalf Of Italo Busi
> Sent: 14 April 2023 15:33
> To: Jeffrey Haas mailto:jh...@pfrc.org>>
> Cc: netmod@ietf.org <mailto:netmod@ietf.org>
> Subject: Re: [netmod] Unknown bits - backwards compatibility
>  
> Hi Jeff,
>  
> I agree with you on the preference for hex-string versus binary in YANG
>  
> I also agree with you concerns with hexdump, but parsing has some issues when 
> something is unknown (these are the issues that have triggered your draft and 
> this discussion), although I also agree with you that the unknown is for 
> ‘exception cases’
>  
> That’s why I would prefer:
> A YANG leaf of type bits to report known bits (after parsing) which will 
> cover “most circumstances”
> Another YANG leaf of type hex-string for the ‘exception cases’ (or for those 
> who really loves hexdump)
>  
> There is a similar issue when reporting a protocol field representing an 
> enumeration. If the received value is known, it would be better to report a 
> string/identity name associated with the recived value but when the value is 
> unknown it is only possible to report the he

Re: [netmod] Unknown bits - backwards compatibility

[Jason Sterne (Nokia)]

That could be one approach, but note that the final step (your last sentence) 
would still be NBC.

From: Rob Wilton (rwilton) 
Sent: Monday, April 17, 2023 9:36 AM
To: Jason Sterne (Nokia) ; Italo Busi 
; Jeffrey Haas 
Cc: netmod@ietf.org
Subject: RE: [netmod] Unknown bits - backwards compatibility


CAUTION: This is an external email. Please be very careful when clicking links 
or opening attachments. See the URL nok.it/ext for additional information.


Hi Jason,

Yes, I was thinking that you would always return the raw value and the bits 
encoded value, but perhaps a server could optionally elide the raw value when 
it knows that it can be fully represented by the bits value.

Regards,
Rob



From: Jason Sterne (Nokia) 
mailto:jason.ste...@nokia.com>>
Sent: 17 April 2023 14:05
To: Rob Wilton (rwilton) mailto:rwil...@cisco.com>>; Italo 
Busi mailto:italo.b...@huawei.com>>; Jeffrey Haas 
mailto:jh...@pfrc.org>>
Cc: netmod@ietf.org<mailto:netmod@ietf.org>
Subject: RE: [netmod] Unknown bits - backwards compatibility

Rob & Italo - are you proposing that the "raw-bits" are all always returned 
(whether they are known or not)?
Jason

From: netmod mailto:netmod-boun...@ietf.org>> On 
Behalf Of Rob Wilton (rwilton)
Sent: Monday, April 17, 2023 7:43 AM
To: Italo Busi 
mailto:Italo.Busi=40huawei@dmarc.ietf.org>>;
 Jeffrey Haas mailto:jh...@pfrc.org>>
Cc: netmod@ietf.org<mailto:netmod@ietf.org>
Subject: Re: [netmod] Unknown bits - backwards compatibility


CAUTION: This is an external email. Please be very careful when clicking links 
or opening attachments. See the URL nok.it/ext for additional information.


Italo's suggest was also how I was thinking of this.

We could define a convention for how the "raw" leaf should be named relative to 
the bits decoded leaf, and also what type the "raw" leaf should use.  E.g., in 
the case where the length of the bits field is known (e.g., it is 
defined/constrained by a protocol spec), then it could just use a uint8 - 
through uint64 as appropriate, otherwise I would have thought that binary would 
be appropriate underlying type.

As new bits get defined then the bits type would get updated, but the "raw" 
leaf wouldn't need to change.

Hence a draft could:

  *   Define a convention for naming the "raw value" leaves to be used in 
conjunction with the decodable bits types leaves.
  *   Define typedefs that wrap (uint8 - uint64, and binary) to indicate the 
semantics as representing undecoded bits.
  *   Define a convention for decoding/displaying the new types (perhaps in 
conjunction with the decoded bits type leaf).

Regards,
Rob

// As a contributor.


From: netmod mailto:netmod-boun...@ietf.org>> On 
Behalf Of Italo Busi
Sent: 14 April 2023 15:33
To: Jeffrey Haas mailto:jh...@pfrc.org>>
Cc: netmod@ietf.org<mailto:netmod@ietf.org>
Subject: Re: [netmod] Unknown bits - backwards compatibility

Hi Jeff,

I agree with you on the preference for hex-string versus binary in YANG

I also agree with you concerns with hexdump, but parsing has some issues when 
something is unknown (these are the issues that have triggered your draft and 
this discussion), although I also agree with you that the unknown is for 
'exception cases'

That's why I would prefer:

  *   A YANG leaf of type bits to report known bits (after parsing) which will 
cover "most circumstances"
  *   Another YANG leaf of type hex-string for the 'exception cases' (or for 
those who really loves hexdump)

There is a similar issue when reporting a protocol field representing an 
enumeration. If the received value is known, it would be better to report a 
string/identity name associated with the recived value but when the value is 
unknown it is only possible to report the hexdump of the field

Italo

From: Jeffrey Haas mailto:jh...@pfrc.org>>
Sent: venerdì 14 aprile 2023 14:49
To: Italo Busi mailto:italo.b...@huawei.com>>
Cc: Jason Sterne (Nokia) 
mailto:jason.ste...@nokia.com>>; 
netmod@ietf.org<mailto:netmod@ietf.org>
Subject: Re: [netmod] Unknown bits - backwards compatibility

Italo,


On Apr 14, 2023, at 4:04 AM, Italo Busi 
mailto:italo.b...@huawei.com>> wrote:

If the need is to report the value of a received protocol field for 
debugging/troubleshooting purposes, I am wondering why not using a binary type 
instead of a bits type or, better, a YANG leaf of bits type for the known bits 
and another YANG leaf of binary type for all known/unknown bits

Type binary isn't generically helpful.  The implementation has to do something 
with the base64 stream it gets.  If it's just going to dump it in hex format, 
you might as well just say that and use hex-string.

The use case here is most bits are known and these are exception cases.  
Ideally in most circumstances, the leaf containing the unknowns is absent.

Perhaps a bit of personal conte

Re: [netmod] Unknown bits - backwards compatibility

[Rob Wilton (rwilton)]

Hi Jason,

Yes, I was thinking that you would always return the raw value and the bits 
encoded value, but perhaps a server could optionally elide the raw value when 
it knows that it can be fully represented by the bits value.

Regards,
Rob



From: Jason Sterne (Nokia) 
Sent: 17 April 2023 14:05
To: Rob Wilton (rwilton) ; Italo Busi 
; Jeffrey Haas 
Cc: netmod@ietf.org
Subject: RE: [netmod] Unknown bits - backwards compatibility

Rob & Italo - are you proposing that the "raw-bits" are all always returned 
(whether they are known or not)?
Jason

From: netmod mailto:netmod-boun...@ietf.org>> On 
Behalf Of Rob Wilton (rwilton)
Sent: Monday, April 17, 2023 7:43 AM
To: Italo Busi 
mailto:Italo.Busi=40huawei@dmarc.ietf.org>>;
 Jeffrey Haas mailto:jh...@pfrc.org>>
Cc: netmod@ietf.org<mailto:netmod@ietf.org>
Subject: Re: [netmod] Unknown bits - backwards compatibility


CAUTION: This is an external email. Please be very careful when clicking links 
or opening attachments. See the URL nok.it/ext for additional information.


Italo's suggest was also how I was thinking of this.

We could define a convention for how the "raw" leaf should be named relative to 
the bits decoded leaf, and also what type the "raw" leaf should use.  E.g., in 
the case where the length of the bits field is known (e.g., it is 
defined/constrained by a protocol spec), then it could just use a uint8 - 
through uint64 as appropriate, otherwise I would have thought that binary would 
be appropriate underlying type.

As new bits get defined then the bits type would get updated, but the "raw" 
leaf wouldn't need to change.

Hence a draft could:

  *   Define a convention for naming the "raw value" leaves to be used in 
conjunction with the decodable bits types leaves.
  *   Define typedefs that wrap (uint8 - uint64, and binary) to indicate the 
semantics as representing undecoded bits.
  *   Define a convention for decoding/displaying the new types (perhaps in 
conjunction with the decoded bits type leaf).

Regards,
Rob

// As a contributor.


From: netmod mailto:netmod-boun...@ietf.org>> On 
Behalf Of Italo Busi
Sent: 14 April 2023 15:33
To: Jeffrey Haas mailto:jh...@pfrc.org>>
Cc: netmod@ietf.org<mailto:netmod@ietf.org>
Subject: Re: [netmod] Unknown bits - backwards compatibility

Hi Jeff,

I agree with you on the preference for hex-string versus binary in YANG

I also agree with you concerns with hexdump, but parsing has some issues when 
something is unknown (these are the issues that have triggered your draft and 
this discussion), although I also agree with you that the unknown is for 
'exception cases'

That's why I would prefer:

  *   A YANG leaf of type bits to report known bits (after parsing) which will 
cover "most circumstances"
  *   Another YANG leaf of type hex-string for the 'exception cases' (or for 
those who really loves hexdump)

There is a similar issue when reporting a protocol field representing an 
enumeration. If the received value is known, it would be better to report a 
string/identity name associated with the recived value but when the value is 
unknown it is only possible to report the hexdump of the field

Italo

From: Jeffrey Haas mailto:jh...@pfrc.org>>
Sent: venerdì 14 aprile 2023 14:49
To: Italo Busi mailto:italo.b...@huawei.com>>
Cc: Jason Sterne (Nokia) 
mailto:jason.ste...@nokia.com>>; 
netmod@ietf.org<mailto:netmod@ietf.org>
Subject: Re: [netmod] Unknown bits - backwards compatibility

Italo,


On Apr 14, 2023, at 4:04 AM, Italo Busi 
mailto:italo.b...@huawei.com>> wrote:

If the need is to report the value of a received protocol field for 
debugging/troubleshooting purposes, I am wondering why not using a binary type 
instead of a bits type or, better, a YANG leaf of bits type for the known bits 
and another YANG leaf of binary type for all known/unknown bits

Type binary isn't generically helpful.  The implementation has to do something 
with the base64 stream it gets.  If it's just going to dump it in hex format, 
you might as well just say that and use hex-string.

The use case here is most bits are known and these are exception cases.  
Ideally in most circumstances, the leaf containing the unknowns is absent.

Perhaps a bit of personal context:
"Hey, Jeff... why not just get a hexdump of the packet?"
"I've spent 20+ years looking at different tools' dumps of bits of BGP PDU and 
I'm sick of it.  The code knows how to parse out fields, let it do that work."

-- Jeff (who keeps expecting this conversation to devolve to a proposal for 
netconf streaming tcpdump)

___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Jason Sterne (Nokia)]

Rob & Italo - are you proposing that the "raw-bits" are all always returned 
(whether they are known or not)?
Jason

From: netmod  On Behalf Of Rob Wilton (rwilton)
Sent: Monday, April 17, 2023 7:43 AM
To: Italo Busi ; Jeffrey Haas 

Cc: netmod@ietf.org
Subject: Re: [netmod] Unknown bits - backwards compatibility


CAUTION: This is an external email. Please be very careful when clicking links 
or opening attachments. See the URL nok.it/ext for additional information.


Italo's suggest was also how I was thinking of this.

We could define a convention for how the "raw" leaf should be named relative to 
the bits decoded leaf, and also what type the "raw" leaf should use.  E.g., in 
the case where the length of the bits field is known (e.g., it is 
defined/constrained by a protocol spec), then it could just use a uint8 - 
through uint64 as appropriate, otherwise I would have thought that binary would 
be appropriate underlying type.

As new bits get defined then the bits type would get updated, but the "raw" 
leaf wouldn't need to change.

Hence a draft could:

  *   Define a convention for naming the "raw value" leaves to be used in 
conjunction with the decodable bits types leaves.
  *   Define typedefs that wrap (uint8 - uint64, and binary) to indicate the 
semantics as representing undecoded bits.
  *   Define a convention for decoding/displaying the new types (perhaps in 
conjunction with the decoded bits type leaf).

Regards,
Rob

// As a contributor.


From: netmod mailto:netmod-boun...@ietf.org>> On 
Behalf Of Italo Busi
Sent: 14 April 2023 15:33
To: Jeffrey Haas mailto:jh...@pfrc.org>>
Cc: netmod@ietf.org<mailto:netmod@ietf.org>
Subject: Re: [netmod] Unknown bits - backwards compatibility

Hi Jeff,

I agree with you on the preference for hex-string versus binary in YANG

I also agree with you concerns with hexdump, but parsing has some issues when 
something is unknown (these are the issues that have triggered your draft and 
this discussion), although I also agree with you that the unknown is for 
'exception cases'

That's why I would prefer:

  *   A YANG leaf of type bits to report known bits (after parsing) which will 
cover "most circumstances"
  *   Another YANG leaf of type hex-string for the 'exception cases' (or for 
those who really loves hexdump)

There is a similar issue when reporting a protocol field representing an 
enumeration. If the received value is known, it would be better to report a 
string/identity name associated with the recived value but when the value is 
unknown it is only possible to report the hexdump of the field

Italo

From: Jeffrey Haas mailto:jh...@pfrc.org>>
Sent: venerdì 14 aprile 2023 14:49
To: Italo Busi mailto:italo.b...@huawei.com>>
Cc: Jason Sterne (Nokia) 
mailto:jason.ste...@nokia.com>>; 
netmod@ietf.org<mailto:netmod@ietf.org>
Subject: Re: [netmod] Unknown bits - backwards compatibility

Italo,


On Apr 14, 2023, at 4:04 AM, Italo Busi 
mailto:italo.b...@huawei.com>> wrote:

If the need is to report the value of a received protocol field for 
debugging/troubleshooting purposes, I am wondering why not using a binary type 
instead of a bits type or, better, a YANG leaf of bits type for the known bits 
and another YANG leaf of binary type for all known/unknown bits

Type binary isn't generically helpful.  The implementation has to do something 
with the base64 stream it gets.  If it's just going to dump it in hex format, 
you might as well just say that and use hex-string.

The use case here is most bits are known and these are exception cases.  
Ideally in most circumstances, the leaf containing the unknowns is absent.

Perhaps a bit of personal context:
"Hey, Jeff... why not just get a hexdump of the packet?"
"I've spent 20+ years looking at different tools' dumps of bits of BGP PDU and 
I'm sick of it.  The code knows how to parse out fields, let it do that work."

-- Jeff (who keeps expecting this conversation to devolve to a proposal for 
netconf streaming tcpdump)

___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Jason Sterne (Nokia)]

In the current proposal/draft, when bit-1 changes to "known", then suddenly the 
server stops sending bit-1 in the unknown-bits. 

That behavior changes whether the author remembers (or decides) to update the 
description of the unknown-flags or not.

I've suggested that the description of the unknown-flags leaf is updated in 
this case.

If the author forgets (or decides not) to update the description, I'm not sure 
we'd want to suddenly say that's backwards compatible. The author should (or 
perhaps MUST) mark the revision as non-backwards-compatible IMO.

Jason

> -Original Message-
> From: Martin Björklund 
> Sent: Monday, April 17, 2023 2:08 AM
> To: Jason Sterne (Nokia) 
> Cc: jh...@pfrc.org; netmod@ietf.org
> Subject: Re: [netmod] Unknown bits - backwards compatibility
> 
> 
> CAUTION: This is an external email. Please be very careful when clicking 
> links or
> opening attachments. See the URL nok.it/ext for additional information.
> 
> 
> 
> Hi,
> 
> 
> "Jason Sterne (Nokia)"  wrote:
> > Hi Martin,
> >
> > If a "description" of a leaf (without a default statement) changes from 
> > this:
> >
> >   "the absence of this leaf causes the protocol to stay 
> > administratively down"
> >
> > to this:
> >
> >   "the absence of this leaf causes the protocol to go administratively 
> > up"
> >
> > (with no other changes in the YANG) then IMO it *is* an NBC
> > change. The behavior described in a "description" field is
> > considered part of the model/API (I've seen many references &
> > examples of this over the years in NETMOD/NETCONF discussions).
> 
> Absolutely.
> 
> But that's not what we disuss in this case.  In this case we have
> a leaf that shows "unknown bits", and neither the type nor the leaf
> (including descriptions) change during the upgrade.
> 
> 
> 
> /martin
> 
> 
> 
> >
> > Maybe it becomes more subtle if that behavior change isn't documented in the
> "description" statement (I'd argue it is still NBC if the server changes that
> behavior and they should be publishing a new revision of the YANG model/API),
> but I was proposing that it should in this case.
> >
> > Jason
> >
> > > -Original Message-
> > > From: Martin Björklund 
> > > Sent: Friday, April 14, 2023 4:39 AM
> > > To: Jason Sterne (Nokia) 
> > > Cc: jh...@pfrc.org; netmod@ietf.org
> > > Subject: Re: [netmod] Unknown bits - backwards compatibility
> > >
> > >
> > > CAUTION: This is an external email. Please be very careful when clicking 
> > > links
> or
> > > opening attachments. See the URL nok.it/ext for additional information.
> > >
> > >
> > >
> > > Hi,
> > >
> > > I am quite confused after reading this thread, so I had to go back to
> > > this first message:
> > >
> > > "Jason Sterne (Nokia)"  wrote:
> > > > Hi Jeff,
> > > >
> > > > One topic that came up during the IETF 116 NETMOD meeting was
> > > > backwards compatibility.
> > > >
> > > > >From what I understand, a leaf (e.g. unknown-flags) that uses the
> > > > >unknown-bits typedef would never change its definition in YANG. It
> > > > >would always be defined as unknown-bits with all 64 bit definitions
> > > > >even as more and more bits become "known".  *But* the system would
> > > > >suddenly stop reporting bit-0, then bit-1 in that unknown-flags leaf
> > > > >as those bits become known.
> > > >
> > > > Strictly speaking, that should probably be considered an NBC change
> > >
> > > Nothing has changed in the data model, so there is no way to mark the
> > > _data model_ as NBC.
> > >
> > > The server follows the data model, and reports which bits it doesn't
> > > understand.  With software updates, this may change over time.  This
> > > is simply the semantics of this state leaf.
> > >
> > >
> > > /martin

___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Rob Wilton (rwilton)]

Italo's suggest was also how I was thinking of this.

We could define a convention for how the "raw" leaf should be named relative to 
the bits decoded leaf, and also what type the "raw" leaf should use.  E.g., in 
the case where the length of the bits field is known (e.g., it is 
defined/constrained by a protocol spec), then it could just use a uint8 - 
through uint64 as appropriate, otherwise I would have thought that binary would 
be appropriate underlying type.

As new bits get defined then the bits type would get updated, but the "raw" 
leaf wouldn't need to change.

Hence a draft could:

  *   Define a convention for naming the "raw value" leaves to be used in 
conjunction with the decodable bits types leaves.
  *   Define typedefs that wrap (uint8 - uint64, and binary) to indicate the 
semantics as representing undecoded bits.
  *   Define a convention for decoding/displaying the new types (perhaps in 
conjunction with the decoded bits type leaf).

Regards,
Rob

// As a contributor.


From: netmod  On Behalf Of Italo Busi
Sent: 14 April 2023 15:33
To: Jeffrey Haas 
Cc: netmod@ietf.org
Subject: Re: [netmod] Unknown bits - backwards compatibility

Hi Jeff,

I agree with you on the preference for hex-string versus binary in YANG

I also agree with you concerns with hexdump, but parsing has some issues when 
something is unknown (these are the issues that have triggered your draft and 
this discussion), although I also agree with you that the unknown is for 
'exception cases'

That's why I would prefer:

  *   A YANG leaf of type bits to report known bits (after parsing) which will 
cover "most circumstances"
  *   Another YANG leaf of type hex-string for the 'exception cases' (or for 
those who really loves hexdump)

There is a similar issue when reporting a protocol field representing an 
enumeration. If the received value is known, it would be better to report a 
string/identity name associated with the recived value but when the value is 
unknown it is only possible to report the hexdump of the field

Italo

From: Jeffrey Haas mailto:jh...@pfrc.org>>
Sent: venerdì 14 aprile 2023 14:49
To: Italo Busi mailto:italo.b...@huawei.com>>
Cc: Jason Sterne (Nokia) 
mailto:jason.ste...@nokia.com>>; 
netmod@ietf.org<mailto:netmod@ietf.org>
Subject: Re: [netmod] Unknown bits - backwards compatibility

Italo,


On Apr 14, 2023, at 4:04 AM, Italo Busi 
mailto:italo.b...@huawei.com>> wrote:

If the need is to report the value of a received protocol field for 
debugging/troubleshooting purposes, I am wondering why not using a binary type 
instead of a bits type or, better, a YANG leaf of bits type for the known bits 
and another YANG leaf of binary type for all known/unknown bits

Type binary isn't generically helpful.  The implementation has to do something 
with the base64 stream it gets.  If it's just going to dump it in hex format, 
you might as well just say that and use hex-string.

The use case here is most bits are known and these are exception cases.  
Ideally in most circumstances, the leaf containing the unknowns is absent.

Perhaps a bit of personal context:
"Hey, Jeff... why not just get a hexdump of the packet?"
"I've spent 20+ years looking at different tools' dumps of bits of BGP PDU and 
I'm sick of it.  The code knows how to parse out fields, let it do that work."

-- Jeff (who keeps expecting this conversation to devolve to a proposal for 
netconf streaming tcpdump)

___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Martin Björklund]

Hi,


"Jason Sterne (Nokia)"  wrote:
> Hi Martin,
> 
> If a "description" of a leaf (without a default statement) changes from this:
> 
>   "the absence of this leaf causes the protocol to stay administratively 
> down"
> 
> to this:
> 
>   "the absence of this leaf causes the protocol to go administratively up"
> 
> (with no other changes in the YANG) then IMO it *is* an NBC
> change. The behavior described in a "description" field is
> considered part of the model/API (I've seen many references &
> examples of this over the years in NETMOD/NETCONF discussions).

Absolutely.

But that's not what we disuss in this case.  In this case we have
a leaf that shows "unknown bits", and neither the type nor the leaf
(including descriptions) change during the upgrade.



/martin



> 
> Maybe it becomes more subtle if that behavior change isn't documented in the 
> "description" statement (I'd argue it is still NBC if the server changes that 
> behavior and they should be publishing a new revision of the YANG model/API), 
> but I was proposing that it should in this case.
> 
> Jason
> 
> > -Original Message-
> > From: Martin Björklund 
> > Sent: Friday, April 14, 2023 4:39 AM
> > To: Jason Sterne (Nokia) 
> > Cc: jh...@pfrc.org; netmod@ietf.org
> > Subject: Re: [netmod] Unknown bits - backwards compatibility
> > 
> > 
> > CAUTION: This is an external email. Please be very careful when clicking 
> > links or
> > opening attachments. See the URL nok.it/ext for additional information.
> > 
> > 
> > 
> > Hi,
> > 
> > I am quite confused after reading this thread, so I had to go back to
> > this first message:
> > 
> > "Jason Sterne (Nokia)"  wrote:
> > > Hi Jeff,
> > >
> > > One topic that came up during the IETF 116 NETMOD meeting was
> > > backwards compatibility.
> > >
> > > >From what I understand, a leaf (e.g. unknown-flags) that uses the
> > > >unknown-bits typedef would never change its definition in YANG. It
> > > >would always be defined as unknown-bits with all 64 bit definitions
> > > >even as more and more bits become "known".  *But* the system would
> > > >suddenly stop reporting bit-0, then bit-1 in that unknown-flags leaf
> > > >as those bits become known.
> > >
> > > Strictly speaking, that should probably be considered an NBC change
> > 
> > Nothing has changed in the data model, so there is no way to mark the
> > _data model_ as NBC.
> > 
> > The server follows the data model, and reports which bits it doesn't
> > understand.  With software updates, this may change over time.  This
> > is simply the semantics of this state leaf.
> > 
> > 
> > /martin

___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Andy Bierman]

On Fri, Apr 14, 2023 at 11:45 AM Jürgen Schönwälder
 wrote:

> On Fri, Apr 14, 2023 at 03:23:03PM +, Jason Sterne (Nokia) wrote:
>
> > IETF and vendor models are already doing NBC changes. The versioning
> > work is mostly just adding a way to indicate that to users/clients
> > when it happens.
>
> An industry proposal to replace the versioning rules of a versioned
> data modeling language without bumping the version number of the data
> modeling language makes me wonder whether the industry is ready to
> take versioning rules serious.
>
>
:-)

Adding new requirements to YANG that the major version
number MUST be incremented when making NBC changes,
while making NBC changes to YANG without even bumping the version number.



> /js
>

Andy


> --
> Jürgen Schönwälder  Constructor University Bremen gGmbH
> Phone: +49 421 200 3587 Campus Ring 1 | 28759 Bremen | Germany
> Fax:   +49 421 200 3103 
>
___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Jason Sterne (Nokia)]

Hi Jürgen,

Maybe that would be ideal, but from a practical POV this will be helpful now.  
Authors in various organizations occasionally have to make NBC changes in their 
YANG 1.0 and YANG 1.1 modules and we need a way to alert users that has 
happened.

Jason

> -Original Message-
> From: Jürgen Schönwälder 
> Sent: Friday, April 14, 2023 2:46 PM
> To: Jason Sterne (Nokia) 
> Cc: Andy Bierman ; netmod@ietf.org
> Subject: Re: [netmod] Unknown bits - backwards compatibility
> 
> 
> CAUTION: This is an external email. Please be very careful when clicking 
> links or
> opening attachments. See the URL nok.it/ext for additional information.
> 
> 
> 
> On Fri, Apr 14, 2023 at 03:23:03PM +, Jason Sterne (Nokia) wrote:
> 
> > IETF and vendor models are already doing NBC changes. The versioning
> > work is mostly just adding a way to indicate that to users/clients
> > when it happens.
> 
> An industry proposal to replace the versioning rules of a versioned
> data modeling language without bumping the version number of the data
> modeling language makes me wonder whether the industry is ready to
> take versioning rules serious.
> 
> /js
> --
> Jürgen Schönwälder  Constructor University Bremen gGmbH
> Phone: +49 421 200 3587 Campus Ring 1 | 28759 Bremen | Germany
> Fax:   +49 421 200 3103 <https://constructor.university/>
___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Jürgen Schönwälder]

On Fri, Apr 14, 2023 at 03:23:03PM +, Jason Sterne (Nokia) wrote:

> IETF and vendor models are already doing NBC changes. The versioning
> work is mostly just adding a way to indicate that to users/clients
> when it happens.

An industry proposal to replace the versioning rules of a versioned
data modeling language without bumping the version number of the data
modeling language makes me wonder whether the industry is ready to
take versioning rules serious.

/js
-- 
Jürgen Schönwälder  Constructor University Bremen gGmbH
Phone: +49 421 200 3587 Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103 

___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Andy Bierman]

On Fri, Apr 14, 2023 at 7:33 AM Italo Busi  wrote:

> Hi Jeff,
>
>
>
> I agree with you on the preference for hex-string versus binary in YANG
>
>
>
> I also agree with you concerns with hexdump, but parsing has some issues
> when something is unknown (these are the issues that have triggered your
> draft and this discussion), although I also agree with you that the unknown
> is for ‘exception cases’
>
>
>
> That’s why I would prefer:
>
>- A YANG leaf of type bits to report known bits (after parsing) which
>will cover “most circumstances”
>- Another YANG leaf of type hex-string for the ‘exception cases’ (or
>for those who really loves hexdump)
>
>
>


Maybe because hexdump is over 20X more efficient on the wire than the bits
proposal.
Or that a special custom typedef is not needed for every leaf to represent
the bit names.
Or that an extra bit-name to bit-position lookup is not required.

The bits type is intended for permanent bit assignments, where
there is semantics associated with each bit (other than an unknown bit N
set).
It is not meant for temporary assignments that change the identifier over
time.

It is possible to change the status of a bit to "obsolete", which seems to
be sufficient
for the unknown-bits use-case.


There is a similar issue when reporting a protocol field representing an
> enumeration. If the received value is known, it would be better to report a
> string/identity name associated with the recived value but when the value
> is unknown it is only possible to report the hexdump of the field
>
>
>
> Italo
>

Andy


>
>
> *From:* Jeffrey Haas 
> *Sent:* venerdì 14 aprile 2023 14:49
> *To:* Italo Busi 
> *Cc:* Jason Sterne (Nokia) ; netmod@ietf.org
> *Subject:* Re: [netmod] Unknown bits - backwards compatibility
>
>
>
> Italo,
>
>
>
>
>
> On Apr 14, 2023, at 4:04 AM, Italo Busi  wrote:
>
>
>
> If the need is to report the value of a received protocol field for
> debugging/troubleshooting purposes, I am wondering why not using a binary
> type instead of a bits type or, better, a YANG leaf of bits type for the
> known bits and another YANG leaf of binary type for all known/unknown bits
>
>
>
> Type binary isn't generically helpful.  The implementation has to do
> something with the base64 stream it gets.  If it's just going to dump it in
> hex format, you might as well just say that and use hex-string.
>
>
>
> The use case here is most bits are known and these are exception cases.
> Ideally in most circumstances, the leaf containing the unknowns is absent.
>
>
>
> Perhaps a bit of personal context:
>
> "Hey, Jeff... why not just get a hexdump of the packet?"
>
> "I've spent 20+ years looking at different tools' dumps of bits of BGP PDU
> and I'm sick of it.  The code knows how to parse out fields, let it do that
> work."
>
>
>
> -- Jeff (who keeps expecting this conversation to devolve to a proposal
> for netconf streaming tcpdump)
>
>
> ___
> netmod mailing list
> netmod@ietf.org
> https://www.ietf.org/mailman/listinfo/netmod
>
___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Jason Sterne (Nokia)]

In either case, when they happen, it is better to mark it in some standard way 
than do it silently IMO.

From: Andy Bierman 
Sent: Friday, April 14, 2023 12:04 PM
To: Jason Sterne (Nokia) 
Cc: Jürgen Schönwälder ; netmod@ietf.org
Subject: Re: [netmod] Unknown bits - backwards compatibility


CAUTION: This is an external email. Please be very careful when clicking links 
or opening attachments. See the URL nok.it/ext for additional information.




On Fri, Apr 14, 2023 at 8:23 AM Jason Sterne (Nokia) 
mailto:jason.ste...@nokia.com>> wrote:
IETF and vendor models are already doing NBC changes. The versioning work is 
mostly just adding a way to indicate that to users/clients when it happens.


Yes. And all such changes are non-conforming.
It is one thing to make an NBC change to fix a mistake.
Quite another to use NBC changes are part of the data model design.


Jason

Andy


> -Original Message-
> From: Jürgen Schönwälder 
> mailto:jschoenwaelder@constructor.university>>
> Sent: Friday, April 14, 2023 2:12 AM
> To: Andy Bierman mailto:a...@yumaworks.com>>
> Cc: Jason Sterne (Nokia) 
> mailto:jason.ste...@nokia.com>>; 
> netmod@ietf.org<mailto:netmod@ietf.org>
> Subject: Re: [netmod] Unknown bits - backwards compatibility
>
>
> CAUTION: This is an external email. Please be very careful when clicking 
> links or
> opening attachments. See the URL nok.it/ext<http://nok.it/ext> for additional 
> information.
>
>
>
> On Thu, Apr 13, 2023 at 02:32:28PM -0700, Andy Bierman wrote:
> >
> > I do not see any way to interpret RFC 7950 such that a YANG
> > extension can be added later to another document that overrides any
> > normative behavior defined in RFC 7950.
> >
> > So as long as a vendor wants to claim conformance to YANG 1.1, no
> > MUSTs in 7950 can be violated. Period.  That may be harsh, but MUST
> > and MUST NOT work that way.
>
> +1 (even though we may be getting off topic here)
>
> /js
>
> --
> Jürgen Schönwälder  Constructor University Bremen gGmbH
> Phone: +49 421 200 3587 Campus Ring 1 | 28759 Bremen | Germany
> Fax:   +49 421 200 3103 <https://constructor.university/>
___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Jeffrey Haas]

> On Apr 14, 2023, at 10:55 AM, Jürgen Schönwälder 
>  wrote:
> 
> On Fri, Apr 14, 2023 at 08:48:39AM -0400, Jeffrey Haas wrote:
>> 
>> -- Jeff (who keeps expecting this conversation to devolve to a proposal for 
>> netconf streaming tcpdump)
>> 
> 
> So let me help with that.

I should have avoided typing this. :-)

> The allocation of unused bits is probably
> the most basic protocol extension you can think of. So what do we do
> if a protocol engine receives messages of unknown message type?  To
> expose the details of the unknown message, we indeed need a proposal
> for streaming pcap data over netconf. And in case things are messed up
> internally by the software, perhaps also an associated memory dump
> protocol to aid with software debugging?

A flavor of this is present in OSPF's modeling for its raw types.  But in such 
cases, you're getting a full dump of the element.

This isn't my use case.  This is a known syntactically valid bit vector.  
There's no need for a full-packet dump as your first line of debugging.

As a very new operator at a small tier-3 ISP, my first introduction to PDU 
decoding was turning on hexdump tracing for Livingston Portmaster CHAP and ISDN 
sessions.  I have no desire for this to be the default way we push people to 
model their debugging experiences for data that is largely known.

When things have truly gone to hell, raw packets are helpful.  Perhaps a 
different effort for enabling exception debugging is a worthy netconf extension.

-- Jeff

___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Andy Bierman]

On Fri, Apr 14, 2023 at 8:23 AM Jason Sterne (Nokia) 
wrote:

> IETF and vendor models are already doing NBC changes. The versioning work
> is mostly just adding a way to indicate that to users/clients when it
> happens.
>


Yes. And all such changes are non-conforming.
It is one thing to make an NBC change to fix a mistake.
Quite another to use NBC changes are part of the data model design.


Jason
>

Andy


>
> > -Original Message-
> > From: Jürgen Schönwälder 
> > Sent: Friday, April 14, 2023 2:12 AM
> > To: Andy Bierman 
> > Cc: Jason Sterne (Nokia) ; netmod@ietf.org
> > Subject: Re: [netmod] Unknown bits - backwards compatibility
> >
> >
> > CAUTION: This is an external email. Please be very careful when clicking
> links or
> > opening attachments. See the URL nok.it/ext for additional information.
> >
> >
> >
> > On Thu, Apr 13, 2023 at 02:32:28PM -0700, Andy Bierman wrote:
> > >
> > > I do not see any way to interpret RFC 7950 such that a YANG
> > > extension can be added later to another document that overrides any
> > > normative behavior defined in RFC 7950.
> > >
> > > So as long as a vendor wants to claim conformance to YANG 1.1, no
> > > MUSTs in 7950 can be violated. Period.  That may be harsh, but MUST
> > > and MUST NOT work that way.
> >
> > +1 (even though we may be getting off topic here)
> >
> > /js
> >
> > --
> > Jürgen Schönwälder  Constructor University Bremen gGmbH
> > Phone: +49 421 200 3587 Campus Ring 1 | 28759 Bremen | Germany
> > Fax:   +49 421 200 3103 <https://constructor.university/>
>
___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Jeffrey Haas]

Italo,

> On Apr 14, 2023, at 10:32 AM, Italo Busi  wrote:
> I also agree with you concerns with hexdump, but parsing has some issues when 
> something is unknown (these are the issues that have triggered your draft and 
> this discussion), although I also agree with you that the unknown is for 
> ‘exception cases’
>  
> That’s why I would prefer:
> A YANG leaf of type bits to report known bits (after parsing) which will 
> cover “most circumstances”
> Another YANG leaf of type hex-string for the ‘exception cases’ (or for those 
> who really loves hexdump)

What do you think goes into such hex-strings?  And why is that different than 
raw format?

> There is a similar issue when reporting a protocol field representing an 
> enumeration. If the received value is known, it would be better to report a 
> string/identity name associated with the recived value but when the value is 
> unknown it is only possible to report the hexdump of the field

Partial agreement here.  This overlaps the desire to have bits be able to print 
things by default.  Please see the github entry Kent referenced earlier.

-- Jeff

___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Acee Lindem]

> On Apr 14, 2023, at 11:23 AM, Jason Sterne (Nokia)  
> wrote:
> 
> IETF and vendor models are already doing NBC changes. The versioning work is 
> mostly just adding a way to indicate that to users/clients when it happens.

But with new versions of the complete model and not incremental augmentations - 
right? Although I haven’t looked at this for a long time, the YANG tooling will 
flag these as NBC



Thanks,
Acee



> Jason
> 
>> -Original Message-
>> From: Jürgen Schönwälder 
>> Sent: Friday, April 14, 2023 2:12 AM
>> To: Andy Bierman 
>> Cc: Jason Sterne (Nokia) ; netmod@ietf.org
>> Subject: Re: [netmod] Unknown bits - backwards compatibility
>> 
>> 
>> CAUTION: This is an external email. Please be very careful when clicking 
>> links or
>> opening attachments. See the URL nok.it/ext for additional information.
>> 
>> 
>> 
>> On Thu, Apr 13, 2023 at 02:32:28PM -0700, Andy Bierman wrote:
>>> 
>>> I do not see any way to interpret RFC 7950 such that a YANG
>>> extension can be added later to another document that overrides any
>>> normative behavior defined in RFC 7950.
>>> 
>>> So as long as a vendor wants to claim conformance to YANG 1.1, no
>>> MUSTs in 7950 can be violated. Period.  That may be harsh, but MUST
>>> and MUST NOT work that way.
>> 
>> +1 (even though we may be getting off topic here)
>> 
>> /js
>> 
>> --
>> Jürgen Schönwälder  Constructor University Bremen gGmbH
>> Phone: +49 421 200 3587 Campus Ring 1 | 28759 Bremen | Germany
>> Fax:   +49 421 200 3103 <https://constructor.university/>
> ___
> netmod mailing list
> netmod@ietf.org
> https://www.ietf.org/mailman/listinfo/netmod

___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Jason Sterne (Nokia)]

IETF and vendor models are already doing NBC changes. The versioning work is 
mostly just adding a way to indicate that to users/clients when it happens.
Jason

> -Original Message-
> From: Jürgen Schönwälder 
> Sent: Friday, April 14, 2023 2:12 AM
> To: Andy Bierman 
> Cc: Jason Sterne (Nokia) ; netmod@ietf.org
> Subject: Re: [netmod] Unknown bits - backwards compatibility
> 
> 
> CAUTION: This is an external email. Please be very careful when clicking 
> links or
> opening attachments. See the URL nok.it/ext for additional information.
> 
> 
> 
> On Thu, Apr 13, 2023 at 02:32:28PM -0700, Andy Bierman wrote:
> >
> > I do not see any way to interpret RFC 7950 such that a YANG
> > extension can be added later to another document that overrides any
> > normative behavior defined in RFC 7950.
> >
> > So as long as a vendor wants to claim conformance to YANG 1.1, no
> > MUSTs in 7950 can be violated. Period.  That may be harsh, but MUST
> > and MUST NOT work that way.
> 
> +1 (even though we may be getting off topic here)
> 
> /js
> 
> --
> Jürgen Schönwälder  Constructor University Bremen gGmbH
> Phone: +49 421 200 3587 Campus Ring 1 | 28759 Bremen | Germany
> Fax:   +49 421 200 3103 <https://constructor.university/>
___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Jason Sterne (Nokia)]

Hi Martin,

If a "description" of a leaf (without a default statement) changes from this:

"the absence of this leaf causes the protocol to stay administratively 
down"

to this:

"the absence of this leaf causes the protocol to go administratively up"

(with no other changes in the YANG) then IMO it *is* an NBC change. The 
behavior described in a "description" field is considered part of the model/API 
(I've seen many references & examples of this over the years in NETMOD/NETCONF 
discussions).

Maybe it becomes more subtle if that behavior change isn't documented in the 
"description" statement (I'd argue it is still NBC if the server changes that 
behavior and they should be publishing a new revision of the YANG model/API), 
but I was proposing that it should in this case.

Jason

> -Original Message-
> From: Martin Björklund 
> Sent: Friday, April 14, 2023 4:39 AM
> To: Jason Sterne (Nokia) 
> Cc: jh...@pfrc.org; netmod@ietf.org
> Subject: Re: [netmod] Unknown bits - backwards compatibility
> 
> 
> CAUTION: This is an external email. Please be very careful when clicking 
> links or
> opening attachments. See the URL nok.it/ext for additional information.
> 
> 
> 
> Hi,
> 
> I am quite confused after reading this thread, so I had to go back to
> this first message:
> 
> "Jason Sterne (Nokia)"  wrote:
> > Hi Jeff,
> >
> > One topic that came up during the IETF 116 NETMOD meeting was
> > backwards compatibility.
> >
> > >From what I understand, a leaf (e.g. unknown-flags) that uses the
> > >unknown-bits typedef would never change its definition in YANG. It
> > >would always be defined as unknown-bits with all 64 bit definitions
> > >even as more and more bits become "known".  *But* the system would
> > >suddenly stop reporting bit-0, then bit-1 in that unknown-flags leaf
> > >as those bits become known.
> >
> > Strictly speaking, that should probably be considered an NBC change
> 
> Nothing has changed in the data model, so there is no way to mark the
> _data model_ as NBC.
> 
> The server follows the data model, and reports which bits it doesn't
> understand.  With software updates, this may change over time.  This
> is simply the semantics of this state leaf.
> 
> 
> /martin

___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Jürgen Schönwälder]

On Fri, Apr 14, 2023 at 08:48:39AM -0400, Jeffrey Haas wrote:
> 
> -- Jeff (who keeps expecting this conversation to devolve to a proposal for 
> netconf streaming tcpdump)
> 

So let me help with that. The allocation of unused bits is probably
the most basic protocol extension you can think of. So what do we do
if a protocol engine receives messages of unknown message type?  To
expose the details of the unknown message, we indeed need a proposal
for streaming pcap data over netconf. And in case things are messed up
internally by the software, perhaps also an associated memory dump
protocol to aid with software debugging?

/js

-- 
Jürgen Schönwälder  Constructor University Bremen gGmbH
Phone: +49 421 200 3587 Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103 

___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Italo Busi]

Hi Jeff,

I agree with you on the preference for hex-string versus binary in YANG

I also agree with you concerns with hexdump, but parsing has some issues when 
something is unknown (these are the issues that have triggered your draft and 
this discussion), although I also agree with you that the unknown is for 
'exception cases'

That's why I would prefer:

  *   A YANG leaf of type bits to report known bits (after parsing) which will 
cover "most circumstances"
  *   Another YANG leaf of type hex-string for the 'exception cases' (or for 
those who really loves hexdump)

There is a similar issue when reporting a protocol field representing an 
enumeration. If the received value is known, it would be better to report a 
string/identity name associated with the recived value but when the value is 
unknown it is only possible to report the hexdump of the field

Italo

From: Jeffrey Haas 
Sent: venerdì 14 aprile 2023 14:49
To: Italo Busi 
Cc: Jason Sterne (Nokia) ; netmod@ietf.org
Subject: Re: [netmod] Unknown bits - backwards compatibility

Italo,



On Apr 14, 2023, at 4:04 AM, Italo Busi 
mailto:italo.b...@huawei.com>> wrote:

If the need is to report the value of a received protocol field for 
debugging/troubleshooting purposes, I am wondering why not using a binary type 
instead of a bits type or, better, a YANG leaf of bits type for the known bits 
and another YANG leaf of binary type for all known/unknown bits

Type binary isn't generically helpful.  The implementation has to do something 
with the base64 stream it gets.  If it's just going to dump it in hex format, 
you might as well just say that and use hex-string.

The use case here is most bits are known and these are exception cases.  
Ideally in most circumstances, the leaf containing the unknowns is absent.

Perhaps a bit of personal context:
"Hey, Jeff... why not just get a hexdump of the packet?"
"I've spent 20+ years looking at different tools' dumps of bits of BGP PDU and 
I'm sick of it.  The code knows how to parse out fields, let it do that work."

-- Jeff (who keeps expecting this conversation to devolve to a proposal for 
netconf streaming tcpdump)

___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Acee Lindem]

> On Apr 14, 2023, at 07:46, Martin Björklund  wrote:
> 
> Acee Lindem  wrote:
>> 
>>> On Apr 14, 2023, at 04:39, Martin Björklund  wrote:
>>> 
>>> Hi,
>>> 
>>> I am quite confused after reading this thread, so I had to go back to
>>> this first message:
>>> 
>>> "Jason Sterne (Nokia)"  wrote:
 Hi Jeff,
 
 One topic that came up during the IETF 116 NETMOD meeting was
 backwards compatibility.
 
> From what I understand, a leaf (e.g. unknown-flags) that uses the
> unknown-bits typedef would never change its definition in YANG. It
> would always be defined as unknown-bits with all 64 bit definitions
> even as more and more bits become "known".  *But* the system would
> suddenly stop reporting bit-0, then bit-1 in that unknown-flags leaf
> as those bits become known.
 
 Strictly speaking, that should probably be considered an NBC change
>>> 
>>> Nothing has changed in the data model, so there is no way to mark the
>>> _data model_ as NBC.
>>> 
>>> The server follows the data model, and reports which bits it doesn't
>>> understand.  With software updates, this may change over time.  This
>>> is simply the semantics of this state leaf.
>> 
>> I agree. Removing the definition of the unknown bit in the second leaf
>> for unknown bits is not backward compatible but that isn’t being
>> proposed.
>> 
>> Also, one can define a new bit in an unused position in an
>> augmentation - correct?
> 
> No, you cannot change the type of a leaf/leaf-list with augment.
> Augment can only add new nodes.

That’s what I seemed to remember as why we didn’t use bits type bits for the 
IGP models. 
However, I asked ChatGTP to confirm and it led me astray. I’ve marked it as 
incorrect. See response below: 

Yes, YANG 1.1 type bits can be augmented and remain backward compatible as long 
as the augmentation follows certain guidelines.

The YANG 1.1 specification provides guidance on how to augment existing YANG 
data models in a backward-compatible manner. When augmenting a data model that 
uses bits, the augmentation must not redefine any existing bits, but rather can 
only add new bits to the existing bit field.

In addition, the new bits that are added through the augmentation must have a 
unique name within the bit field, and their position within the bit field must 
be chosen so as to not conflict with any existing bits. This ensures that any 
software that uses the original data model will still be able to understand the 
new augmented version of the data model.

It is worth noting that while YANG 1.1 supports backward compatibility, it is 
still important to test the augmented data model thoroughly to ensure that it 
is compatible with all software that may use it.


Thanks,
Acee


> 
> 
> /martin

___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Jeffrey Haas]

Italo,


> On Apr 14, 2023, at 4:04 AM, Italo Busi  wrote:
> 
> If the need is to report the value of a received protocol field for 
> debugging/troubleshooting purposes, I am wondering why not using a binary 
> type instead of a bits type or, better, a YANG leaf of bits type for the 
> known bits and another YANG leaf of binary type for all known/unknown bits

Type binary isn't generically helpful.  The implementation has to do something 
with the base64 stream it gets.  If it's just going to dump it in hex format, 
you might as well just say that and use hex-string.

The use case here is most bits are known and these are exception cases.  
Ideally in most circumstances, the leaf containing the unknowns is absent.

Perhaps a bit of personal context:
"Hey, Jeff... why not just get a hexdump of the packet?"
"I've spent 20+ years looking at different tools' dumps of bits of BGP PDU and 
I'm sick of it.  The code knows how to parse out fields, let it do that work."

-- Jeff (who keeps expecting this conversation to devolve to a proposal for 
netconf streaming tcpdump)

___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[tom petch]

From: netmod  on behalf of Acee Lindem 

Sent: 14 April 2023 12:41

> On Apr 14, 2023, at 04:39, Martin Björklund  wrote:
>
> Hi,
>
> I am quite confused after reading this thread, so I had to go back to
> this first message:
>
> "Jason Sterne (Nokia)"  wrote:
>> Hi Jeff,
>>
>> One topic that came up during the IETF 116 NETMOD meeting was
>> backwards compatibility.
>>
>>> From what I understand, a leaf (e.g. unknown-flags) that uses the
>>> unknown-bits typedef would never change its definition in YANG. It
>>> would always be defined as unknown-bits with all 64 bit definitions
>>> even as more and more bits become "known".  *But* the system would
>>> suddenly stop reporting bit-0, then bit-1 in that unknown-flags leaf
>>> as those bits become known.
>>
>> Strictly speaking, that should probably be considered an NBC change
>
> Nothing has changed in the data model, so there is no way to mark the
> _data model_ as NBC.
>
> The server follows the data model, and reports which bits it doesn't
> understand.  With software updates, this may change over time.  This
> is simply the semantics of this state leaf.

I agree. Removing the definition of the unknown bit in the second leaf for 
unknown bits is not backward compatible but that isn’t being proposed.


Mmm you can apply a restriction so that  a definition of bit4 bit5 bit 6 bit7
becomes just bit6 bit7

Tom Petch

Also, one can define a new bit in an unused position in an augmentation - 
correct?

Thanks,
Acee



>
>
> /martin
>
> ___
> netmod mailing list
> netmod@ietf.org
> https://www.ietf.org/mailman/listinfo/netmod

___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Martin Björklund]

Acee Lindem  wrote:
> 
> > On Apr 14, 2023, at 04:39, Martin Björklund  wrote:
> > 
> > Hi,
> > 
> > I am quite confused after reading this thread, so I had to go back to
> > this first message:
> > 
> > "Jason Sterne (Nokia)"  wrote:
> >> Hi Jeff,
> >> 
> >> One topic that came up during the IETF 116 NETMOD meeting was
> >> backwards compatibility.
> >> 
> >>> From what I understand, a leaf (e.g. unknown-flags) that uses the
> >>> unknown-bits typedef would never change its definition in YANG. It
> >>> would always be defined as unknown-bits with all 64 bit definitions
> >>> even as more and more bits become "known".  *But* the system would
> >>> suddenly stop reporting bit-0, then bit-1 in that unknown-flags leaf
> >>> as those bits become known.
> >> 
> >> Strictly speaking, that should probably be considered an NBC change
> > 
> > Nothing has changed in the data model, so there is no way to mark the
> > _data model_ as NBC.
> > 
> > The server follows the data model, and reports which bits it doesn't
> > understand.  With software updates, this may change over time.  This
> > is simply the semantics of this state leaf.
> 
> I agree. Removing the definition of the unknown bit in the second leaf
> for unknown bits is not backward compatible but that isn’t being
> proposed.
> 
> Also, one can define a new bit in an unused position in an
> augmentation - correct?

No, you cannot change the type of a leaf/leaf-list with augment.
Augment can only add new nodes.


/martin
___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Acee Lindem]

> On Apr 14, 2023, at 04:39, Martin Björklund  wrote:
> 
> Hi,
> 
> I am quite confused after reading this thread, so I had to go back to
> this first message:
> 
> "Jason Sterne (Nokia)"  wrote:
>> Hi Jeff,
>> 
>> One topic that came up during the IETF 116 NETMOD meeting was
>> backwards compatibility.
>> 
>>> From what I understand, a leaf (e.g. unknown-flags) that uses the
>>> unknown-bits typedef would never change its definition in YANG. It
>>> would always be defined as unknown-bits with all 64 bit definitions
>>> even as more and more bits become "known".  *But* the system would
>>> suddenly stop reporting bit-0, then bit-1 in that unknown-flags leaf
>>> as those bits become known.
>> 
>> Strictly speaking, that should probably be considered an NBC change
> 
> Nothing has changed in the data model, so there is no way to mark the
> _data model_ as NBC.
> 
> The server follows the data model, and reports which bits it doesn't
> understand.  With software updates, this may change over time.  This
> is simply the semantics of this state leaf.

I agree. Removing the definition of the unknown bit in the second leaf for 
unknown bits is not backward compatible but that isn’t being proposed. 

Also, one can define a new bit in an unused position in an augmentation - 
correct? 

Thanks,
Acee



> 
> 
> /martin
> 
> ___
> netmod mailing list
> netmod@ietf.org
> https://www.ietf.org/mailman/listinfo/netmod

___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Martin Björklund]

Hi,

I am quite confused after reading this thread, so I had to go back to
this first message:

"Jason Sterne (Nokia)"  wrote:
> Hi Jeff,
> 
> One topic that came up during the IETF 116 NETMOD meeting was
> backwards compatibility.
> 
> >From what I understand, a leaf (e.g. unknown-flags) that uses the
> >unknown-bits typedef would never change its definition in YANG. It
> >would always be defined as unknown-bits with all 64 bit definitions
> >even as more and more bits become "known".  *But* the system would
> >suddenly stop reporting bit-0, then bit-1 in that unknown-flags leaf
> >as those bits become known.
> 
> Strictly speaking, that should probably be considered an NBC change

Nothing has changed in the data model, so there is no way to mark the
_data model_ as NBC.

The server follows the data model, and reports which bits it doesn't
understand.  With software updates, this may change over time.  This
is simply the semantics of this state leaf.


/martin

___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Italo Busi]

Jason, Jeffrey,

If the need is to report the value of a received protocol field for 
debugging/troubleshooting purposes, I am wondering why not using a binary type 
instead of a bits type or, better, a YANG leaf of bits type for the known bits 
and another YANG leaf of binary type for all known/unknown bits

Trailing zeros can be added when the protocol field is not an integer number of 
octets

In this way there will be no backward compatibility issues when unknown bits 
are assigned and becomes known

My 2 cents

Italo

From: Jason Sterne (Nokia) 
Sent: mercoledì 12 aprile 2023 23:26
To: Jason Sterne (Nokia) ; Jeffrey Haas 
; netmod@ietf.org
Subject: Re: [netmod] Unknown bits - backwards compatibility

It just occurred to me that to avoid the NBC change we could also consider just 
calling this new typedef "raw-bits" and always outputting all the bits in the 
second leaf (whether they are known or not)?  I suspect this was already 
considered though...

Jason

From: netmod mailto:netmod-boun...@ietf.org>> On 
Behalf Of Jason Sterne (Nokia)
Sent: Wednesday, April 12, 2023 5:24 PM
To: Jeffrey Haas mailto:jh...@pfrc.org>>; 
netmod@ietf.org<mailto:netmod@ietf.org>
Subject: [netmod] Unknown bits - backwards compatibility

Hi Jeff,

One topic that came up during the IETF 116 NETMOD meeting was backwards 
compatibility.

>From what I understand, a leaf (e.g. unknown-flags) that uses the unknown-bits 
>typedef would never change its definition in YANG. It would always be defined 
>as unknown-bits with all 64 bit definitions even as more and more bits become 
>"known".  *But* the system would suddenly stop reporting bit-0, then bit-1 in 
>that unknown-flags leaf as those bits become known.

Strictly speaking, that should probably be considered an NBC change although it 
is a bit of a grey zone - the NBC rules for state are fuzzy (theoretically they 
are defined by RFC7950 but it is clear those rules were focused on config, and 
the same goes for all our new versioning work). But I could imagine an 
implementation that was previously seeing bit-0 returned and suddenly stops 
seeing that bit-0 returned (which could cause different 
interpretation/behavior). So in some ways the semantics of the unknown-flags 
leaf has changed. It may be better to just flag this as an NBC change so a user 
would be drawn to look at it, and make a decision that they do or don't have to 
update their handling/use of it?

The known flags leaf would only go through backwards compatible changes though 
(since we'd always be additive in adding bits) - assuming bit positions don't 
change in the protocol.

It would probably help to show an example of what a YANG module looks like for 
step 1 and then step 2 (an unknown becomes known), and also what is returned in 
NETCONF in step 1 and step 2. You partially have that in section 3.2 although 
the YANG model isn't shown and it might be good to explicitly mention that 
 isn't returned (note it may also be valid to return 
).

Jason


From: netmod mailto:netmod-boun...@ietf.org>> On 
Behalf Of Jeffrey Haas
Sent: Monday, April 10, 2023 2:48 PM
To: netmod@ietf.org<mailto:netmod@ietf.org>
Subject: Re: [netmod] Request for WG adoption, 
draft-haas-netmod-unknown-bits-01.txt



CAUTION: This is an external email. Please be very careful when clicking links 
or opening attachments. See http://nok.it/ext for additional information.



Netmod Working Group,

The presentation at IETF 116 on the YANG unknown bits typedef seemed to be 
positively received.

I've updated the draft in version -02 to incorporate a hallway suggestion (from 
Rob, I think?) to rename the bits from a pattern of "unknown-N" to "bit-N".  
Please find the information for the updated draft pasted below my original 
request for adoption.

Having given my presentation and seeing there seems to be interest in the work, 
could we consider adoption?

-- Jeff




On Feb 20, 2023, at 1:23 PM, Jeffrey Haas 
mailto:jh...@pfrc.org>> wrote:

The current version of this small draft has addressed the discussion points to 
date.

I'd like to request that the netmod WG consider adopting this draft.  
Alternatively, if it's thought useful but more appropriate to carry this work 
on elsewhere (e.g. rtgwg), I'd appreciate such feedback.

For the process folk, I am unaware of any applicable IPR covering this draft.

-- Jeff


Name:  draft-haas-netmod-unknown-bits
Revision: 02
Title: Representing Unknown YANG bits in Operational State
Document date: 2023-04-10
Group: Individual Submission
Pages: 18
URL:
https://www.ietf.org/archive/id/draft-haas-netmod-unknown-bits-02.txt
Status: https://datatracker.ietf.org/doc/draft-haas-netmod-unknown-bits/
Html:   
https://www.ietf.org/archive/id/draft-haas-netmod-unknown-bits-02.html
Htmlized:   
https://datatracker.ietf.org/doc/html/draft-haas-netmod-unknown-bits
Diff:   
https:/

Re: [netmod] Unknown bits - backwards compatibility

[Jürgen Schönwälder]

On Thu, Apr 13, 2023 at 02:32:28PM -0700, Andy Bierman wrote:
> 
> I do not see any way to interpret RFC 7950 such that a YANG
> extension can be added later to another document that overrides any
> normative behavior defined in RFC 7950.
> 
> So as long as a vendor wants to claim conformance to YANG 1.1, no
> MUSTs in 7950 can be violated. Period.  That may be harsh, but MUST
> and MUST NOT work that way.

+1 (even though we may be getting off topic here)

/js

-- 
Jürgen Schönwälder  Constructor University Bremen gGmbH
Phone: +49 421 200 3587 Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103 

___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Jeffrey Haas]

Jason,


> On Apr 13, 2023, at 5:25 PM, Jason Sterne (Nokia)  
> wrote:
> In the Schema Comparison draft we’re also debating whether to add “per node” 
> compatibility tags (optional – use when needed, and this is a case where it 
> is useful to flag a particular *node* as having an NBC change in version 
> 2.0.0).

I've not read the draft, but this seems to be a Good Thing.  Basically 
"impacted by revX" with a per-rev statement describing that impact.

One thought for accommodating NBC for types such as the unknown-bits one is 
that each instantiation of it in a model simply does its own typedef of that 
typedef.  It thus, as a base, inherits the underlying unknown-bits types, but 
provides a point of maintenance for individual values.

-- Jeff

___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Andy Bierman]

On Thu, Apr 13, 2023 at 2:00 PM Jason Sterne (Nokia) 
wrote:

> Hi Jeff,
>
>
>
> We avoided getting into subtleties about “severity” of an NBC change (hard
> enough to just get agreement on basic rules). But I do think it is useful
> to come up with changes and approaches that have lower impact on
> users/clients even if they are still marked as NBC.
>
>
>
> For the new versioning marking, it would basically be the following. In
> the “revision” statement of the new module:
>
>1. Add "rev:non-backwards-compatible" as per 
> draft-ietf-netmod-yang-module-versioning-08
>- Updated YANG Module Revision Handling
>
> 
>2. Increment the major version of the YANG Semver (i.e. go from 1.0.0
>to 2.0.0) as per draft-ietf-netmod-yang-semver-11 - YANG Semantic
>Versioning
>
>
>
>

I do not see any way to interpret RFC 7950 such that a YANG extension can
be added
later to another document that overrides any normative behavior defined in
RFC 7950.

So as long as a vendor wants to claim conformance to YANG 1.1, no MUSTs in
7950 can
be violated. Period.  That may be harsh, but MUST and MUST NOT work that
way.


That may seem harsh, but we leaned towards being strict on flagging any NBC
> change in order to avoid false-negatives (i.e. false-positives aren’t as
> bad). Users can then diff the module versions, see that only the
> unknown-bits stuff has changed, and then decide if they need to update
> their client, etc.
>
>
>
> Jason
>
>
>


Andy


> *From:* Jeffrey Haas 
> *Sent:* Thursday, April 13, 2023 4:44 PM
> *To:* Jason Sterne (Nokia) 
> *Cc:* netmod@ietf.org
> *Subject:* Re: Unknown bits - backwards compatibility
>
>
>
>
>
> *CAUTION:* This is an external email. Please be very careful when
> clicking links or opening attachments. See the URL nok.it/ext for
> additional information.
>
>
>
> Jason,
>
>
>
>
>
> On Apr 13, 2023, at 4:29 PM, Jason Sterne (Nokia) 
> wrote:
>
> *[>>JTS:] Yeah – I see that perspective. But a client using the old
> API/contract gets new/different behavior for the unknown-flags leaf from a
> new server. Hence NBC – unless we decide in the end to somehow make this
> specific case/typedef an exception but I’m not sure about that yet. The
> typedef and behavior you are describing there may still be useful as-is
> even if we decide to still officially consider the change to unknown-flags
> behavior as NBC (i.e. in new the version of the module that changes a bit
> from unknown to known).*
>
>
>
> Understood.  True to form, I seldom seem to come to netmod with easy
> problems. :-)
>
>
>
> Not having read the current versioning material, this would seem to be a
> case where you could consider this NBC, but likely not-problematic.  I'm
> not sure if you have "severity" as a concept in the discussions thus far.
>
>
>
> If there's current verbiage about documenting NBC considerations, feel
> free to point me to the appropriate documents.  As a general purpose
> mechanism, the draft could simply describe that any time an update to the
> known leaf type is done that the unknown leaf type is flagged for NBC for
> the newly assigned bits.
>
>
>
>
>
> I wish to point you and others concerned on these points to the BGP YANG
> modeling for Extended Communities, which will have these problems in a
> different flavor: Known communities will render via the typedefs, unknown
> will render using the prefix 'raw'.  (See typedef bgp-ext-community-type.)
>  This headache is already a consideration in every BGP implementation that
> deals with extended communities in changing meaning.
>
> *[>>JTS:] Can you point me to a repository or RFC where I can see this?
> I’m not familiar with where this YANG work is being done.*
>
>
>
> Sorry for not including the URL.  This document went to WGLC for IDR a few
> days ago.  We'll be asking (yet yet yet again) for YANG doctor review.
>
> https://www.ietf.org/archive/id/draft-ietf-idr-bgp-model-16.html
>
>
>
> You'll find the typedef issue for extended communities in there, and also
> the field for unknown bits in the operational state that is the genesis for
> this conversation.
>
>
>
> Figure 4 in draft version -02 shows how the type gets modified to add the
> new bit. The model doesn't change between step 1/ step 2 beyond this.  
> *[>>JTS:]
> Sure – maybe just mention explicitly that the model for unknown-flags stays
> unchanged.*
>
>
>
> If you'd find it helpful, I could add to the text covering Figure 8 "after
> assignment, bit-1 is no longer returned in unknown-flags".  Is that what
> you're looking for?*[>>JTS:] Yes – that would probably help.*
>
>
>
> I'll try to roll in these changes soon. Thanks.
>
>
>
> -- Jeff
>
>
> ___
> netmod mailing list
> netmod@ietf.org
> https://www.ietf.org/mailman/listinfo/netmod
>
___

Re: [netmod] Unknown bits - backwards compatibility

[Jason Sterne (Nokia)]

Hi Jeff,

You are absolutely right that this is a tricky case for identifying the 
specific affected node. I should have mentioned how to do that.

Basically in this case, the “description” field of the unknown-flags leaf 
should really change each time there are newly unreported bit-0, bit-1, etc 
bits. This is a situation where the YANG model itself doesn’t have a change for 
known-flags, but the underlying *behavior* of that leaf has changed.

This is one of the reasons why we’re having a debate right now for our “Schema 
Comparison” draft about how we should treat description changes by default 
(i.e. not as just “editorial” but as “potentially-nbc” or just “nbc”).

In the Schema Comparison draft we’re also debating whether to add “per node” 
compatibility tags (optional – use when needed, and this is a case where it is 
useful to flag a particular *node* as having an NBC change in version 2.0.0).

Jason

From: Jeffrey Haas 
Sent: Thursday, April 13, 2023 5:18 PM
To: Jason Sterne (Nokia) 
Cc: netmod@ietf.org
Subject: Re: Unknown bits - backwards compatibility


CAUTION: This is an external email. Please be very careful when clicking links 
or opening attachments. See the URL nok.it/ext for additional information.


Jason,



On Apr 13, 2023, at 5:00 PM, Jason Sterne (Nokia) 
mailto:jason.ste...@nokia.com>> wrote:

Hi Jeff,

We avoided getting into subtleties about “severity” of an NBC change (hard 
enough to just get agreement on basic rules). But I do think it is useful to 
come up with changes and approaches that have lower impact on users/clients 
even if they are still marked as NBC.

For the new versioning marking, it would basically be the following. In the 
“revision” statement of the new module:

  1.  Add "rev:non-backwards-compatible" as per 
draft-ietf-netmod-yang-module-versioning-08 - Updated YANG Module Revision 
Handling
  2.  Increment the major version of the YANG Semver (i.e. go from 1.0.0 to 
2.0.0) as per draft-ietf-netmod-yang-semver-11 - YANG Semantic 
Versioning

That may seem harsh, but we leaned towards being strict on flagging any NBC 
change in order to avoid false-negatives (i.e. false-positives aren’t as bad). 
Users can then diff the module versions, see that only the unknown-bits stuff 
has changed, and then decide if they need to update their client, etc.

I think I'm still unclear on how this would manifest for my proposal.

The "known" field isn't having a NBC change, so I think we can ignore that.

The "unknown" field has a stable type. More importantly, and perhaps a larger 
question for the NBC work, what needs to be flagged is that a specific bit has 
been impacted at a specific version bump of the document.  How is that to be 
signaled on such a type?

Consider instead the non-general case, and one that's in the bgp-model:

leaf unknown-flags {

  type bits {

bit unknown-2 {

  position 2;

  description

"Bit 2 was received but is currently RESERVED.";

}

bit unknown-3 {

  position 3;

  description

"Bit 3 was received but is currently RESERVED.";

}

  }
This represents the modeling from my unknown draft for the remaining two bits 
of the nybble that aren't assigned after the second bit was assigned.

If a future version of BGP's Graceful Restart feature assigned bit 2, and 
unknown-flags wasn't modeled using my proposed typedef, I can see the following 
actions being done:

Add new type to the known leaf.  (A backward compatible change.)
Delete "unknown-2".  This is NBC, but clearly documents the change.

Briefly scrolling through the module-versioning draft, it doesn't appear that 
there's ways to flag what nodes contribute to the NBC issues.

In any case, where the contents are visibly impacted, it's clear you can flag 
this.  I suspect you're missing a way to flag the case as documented in my 
current draft.

-- Jeff

___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Jeffrey Haas]

Jason,


> On Apr 13, 2023, at 5:00 PM, Jason Sterne (Nokia)  
> wrote:
> 
> Hi Jeff,
>  
> We avoided getting into subtleties about “severity” of an NBC change (hard 
> enough to just get agreement on basic rules). But I do think it is useful to 
> come up with changes and approaches that have lower impact on users/clients 
> even if they are still marked as NBC.
>  
> For the new versioning marking, it would basically be the following. In the 
> “revision” statement of the new module:
> Add "rev:non-backwards-compatible" as per 
> draft-ietf-netmod-yang-module-versioning-08 - Updated YANG Module Revision 
> Handling 
> 
> Increment the major version of the YANG Semver (i.e. go from 1.0.0 to 2.0.0) 
> as per draft-ietf-netmod-yang-semver-11 - YANG Semantic Versioning 
> 
>  
> That may seem harsh, but we leaned towards being strict on flagging any NBC 
> change in order to avoid false-negatives (i.e. false-positives aren’t as 
> bad). Users can then diff the module versions, see that only the unknown-bits 
> stuff has changed, and then decide if they need to update their client, etc.

I think I'm still unclear on how this would manifest for my proposal.

The "known" field isn't having a NBC change, so I think we can ignore that.

The "unknown" field has a stable type. More importantly, and perhaps a larger 
question for the NBC work, what needs to be flagged is that a specific bit has 
been impacted at a specific version bump of the document.  How is that to be 
signaled on such a type?

Consider instead the non-general case, and one that's in the bgp-model:
leaf unknown-flags {
  type bits {
bit unknown-2 {
  position 2;
  description
"Bit 2 was received but is currently RESERVED.";
}
bit unknown-3 {
  position 3;
  description
"Bit 3 was received but is currently RESERVED.";
}
  }
This represents the modeling from my unknown draft for the remaining two bits 
of the nybble that aren't assigned after the second bit was assigned.

If a future version of BGP's Graceful Restart feature assigned bit 2, and 
unknown-flags wasn't modeled using my proposed typedef, I can see the following 
actions being done:

Add new type to the known leaf.  (A backward compatible change.)
Delete "unknown-2".  This is NBC, but clearly documents the change.

Briefly scrolling through the module-versioning draft, it doesn't appear that 
there's ways to flag what nodes contribute to the NBC issues.

In any case, where the contents are visibly impacted, it's clear you can flag 
this.  I suspect you're missing a way to flag the case as documented in my 
current draft.

-- Jeff

___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Jason Sterne (Nokia)]

Hi Jeff,

We avoided getting into subtleties about “severity” of an NBC change (hard 
enough to just get agreement on basic rules). But I do think it is useful to 
come up with changes and approaches that have lower impact on users/clients 
even if they are still marked as NBC.

For the new versioning marking, it would basically be the following. In the 
“revision” statement of the new module:

  1.  Add "rev:non-backwards-compatible" as per 
draft-ietf-netmod-yang-module-versioning-08 - Updated YANG Module Revision 
Handling
  2.  Increment the major version of the YANG Semver (i.e. go from 1.0.0 to 
2.0.0) as per draft-ietf-netmod-yang-semver-11 - YANG Semantic 
Versioning

That may seem harsh, but we leaned towards being strict on flagging any NBC 
change in order to avoid false-negatives (i.e. false-positives aren’t as bad). 
Users can then diff the module versions, see that only the unknown-bits stuff 
has changed, and then decide if they need to update their client, etc.

Jason

From: Jeffrey Haas 
Sent: Thursday, April 13, 2023 4:44 PM
To: Jason Sterne (Nokia) 
Cc: netmod@ietf.org
Subject: Re: Unknown bits - backwards compatibility


CAUTION: This is an external email. Please be very careful when clicking links 
or opening attachments. See the URL nok.it/ext for additional information.


Jason,



On Apr 13, 2023, at 4:29 PM, Jason Sterne (Nokia) 
mailto:jason.ste...@nokia.com>> wrote:
[>>JTS:] Yeah – I see that perspective. But a client using the old API/contract 
gets new/different behavior for the unknown-flags leaf from a new server. Hence 
NBC – unless we decide in the end to somehow make this specific case/typedef an 
exception but I’m not sure about that yet. The typedef and behavior you are 
describing there may still be useful as-is even if we decide to still 
officially consider the change to unknown-flags behavior as NBC (i.e. in new 
the version of the module that changes a bit from unknown to known).

Understood.  True to form, I seldom seem to come to netmod with easy problems. 
:-)

Not having read the current versioning material, this would seem to be a case 
where you could consider this NBC, but likely not-problematic.  I'm not sure if 
you have "severity" as a concept in the discussions thus far.

If there's current verbiage about documenting NBC considerations, feel free to 
point me to the appropriate documents.  As a general purpose mechanism, the 
draft could simply describe that any time an update to the known leaf type is 
done that the unknown leaf type is flagged for NBC for the newly assigned bits.


I wish to point you and others concerned on these points to the BGP YANG 
modeling for Extended Communities, which will have these problems in a 
different flavor: Known communities will render via the typedefs, unknown will 
render using the prefix 'raw'.  (See typedef bgp-ext-community-type.)  This 
headache is already a consideration in every BGP implementation that deals with 
extended communities in changing meaning.
[>>JTS:] Can you point me to a repository or RFC where I can see this? I’m not 
familiar with where this YANG work is being done.

Sorry for not including the URL.  This document went to WGLC for IDR a few days 
ago.  We'll be asking (yet yet yet again) for YANG doctor review.
https://www.ietf.org/archive/id/draft-ietf-idr-bgp-model-16.html

You'll find the typedef issue for extended communities in there, and also the 
field for unknown bits in the operational state that is the genesis for this 
conversation.

Figure 4 in draft version -02 shows how the type gets modified to add the new 
bit. The model doesn't change between step 1/ step 2 beyond this.  [>>JTS:] 
Sure – maybe just mention explicitly that the model for unknown-flags stays 
unchanged.

If you'd find it helpful, I could add to the text covering Figure 8 "after 
assignment, bit-1 is no longer returned in unknown-flags".  Is that what you're 
looking for?[>>JTS:] Yes – that would probably help.

I'll try to roll in these changes soon. Thanks.

-- Jeff

___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Jeffrey Haas]

Jason,


> On Apr 13, 2023, at 4:29 PM, Jason Sterne (Nokia)  
> wrote:
> [>>JTS:] Yeah – I see that perspective. But a client using the old 
> API/contract gets new/different behavior for the unknown-flags leaf from a 
> new server. Hence NBC – unless we decide in the end to somehow make this 
> specific case/typedef an exception but I’m not sure about that yet. The 
> typedef and behavior you are describing there may still be useful as-is even 
> if we decide to still officially consider the change to unknown-flags 
> behavior as NBC (i.e. in new the version of the module that changes a bit 
> from unknown to known).

Understood.  True to form, I seldom seem to come to netmod with easy problems. 
:-)

Not having read the current versioning material, this would seem to be a case 
where you could consider this NBC, but likely not-problematic.  I'm not sure if 
you have "severity" as a concept in the discussions thus far.

If there's current verbiage about documenting NBC considerations, feel free to 
point me to the appropriate documents.  As a general purpose mechanism, the 
draft could simply describe that any time an update to the known leaf type is 
done that the unknown leaf type is flagged for NBC for the newly assigned bits.

> 
> I wish to point you and others concerned on these points to the BGP YANG 
> modeling for Extended Communities, which will have these problems in a 
> different flavor: Known communities will render via the typedefs, unknown 
> will render using the prefix 'raw'.  (See typedef bgp-ext-community-type.)  
> This headache is already a consideration in every BGP implementation that 
> deals with extended communities in changing meaning.
> [>>JTS:] Can you point me to a repository or RFC where I can see this? I’m 
> not familiar with where this YANG work is being done.

Sorry for not including the URL.  This document went to WGLC for IDR a few days 
ago.  We'll be asking (yet yet yet again) for YANG doctor review.
https://www.ietf.org/archive/id/draft-ietf-idr-bgp-model-16.html

You'll find the typedef issue for extended communities in there, and also the 
field for unknown bits in the operational state that is the genesis for this 
conversation.

> Figure 4 in draft version -02 shows how the type gets modified to add the new 
> bit. The model doesn't change between step 1/ step 2 beyond this.  [>>JTS:] 
> Sure – maybe just mention explicitly that the model for unknown-flags stays 
> unchanged.
>  
> If you'd find it helpful, I could add to the text covering Figure 8 "after 
> assignment, bit-1 is no longer returned in unknown-flags".  Is that what 
> you're looking for?[>>JTS:] Yes – that would probably help.

I'll try to roll in these changes soon. Thanks.

-- Jeff

___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Jason Sterne (Nokia)]

Hi Jeff,
Please see inline.
Jason

From: Jeffrey Haas 
Sent: Thursday, April 13, 2023 4:03 PM
To: Jason Sterne (Nokia) 
Cc: netmod@ietf.org
Subject: Re: Unknown bits - backwards compatibility


CAUTION: This is an external email. Please be very careful when clicking links 
or opening attachments. See the URL nok.it/ext for additional information.


Jason,



On Apr 12, 2023, at 5:26 PM, Jason Sterne (Nokia) 
mailto:jason.ste...@nokia.com>> wrote:

It just occurred to me that to avoid the NBC change we could also consider just 
calling this new typedef “raw-bits” and always outputting all the bits in the 
second leaf (whether they are known or not)?  I suspect this was already 
considered though…

It was and part of the discussion.  As I mentioned to Andy, consistent advice 
for modeling the raw stuff might be helpful.  However, for bit fields I find 
the likely options to be annoying and probably not good ease of use for the 
end-user.

More below



From: netmod mailto:netmod-boun...@ietf.org>> On 
Behalf Of Jason Sterne (Nokia)


One topic that came up during the IETF 116 NETMOD meeting was backwards 
compatibility.

From what I understand, a leaf (e.g. unknown-flags) that uses the unknown-bits 
typedef would never change its definition in YANG. It would always be defined 
as unknown-bits with all 64 bit definitions even as more and more bits become 
“known”.  *But* the system would suddenly stop reporting bit-0, then bit-1 in 
that unknown-flags leaf as those bits become known.

That's the current proposal, yes.



Strictly speaking, that should probably be considered an NBC change although it 
is a bit of a grey zone - the NBC rules for state are fuzzy (theoretically they 
are defined by RFC7950 but it is clear those rules were focused on config, and 
the same goes for all our new versioning work). But I could imagine an 
implementation that was previously seeing bit-0 returned and suddenly stops 
seeing that bit-0 returned (which could cause different 
interpretation/behavior). So in some ways the semantics of the unknown-flags 
leaf has changed. It may be better to just flag this as an NBC change so a user 
would be drawn to look at it, and make a decision that they do or don’t have to 
update their handling/use of it?

I'm not following the versioning discussions enough to know the full semantics 
you're implying.
[>>JTS:] It is basically the fact that the “old” server returned bit-1 in a 
specific scenario and now the new server doesn’t return bit-1 in that same 
scenario. Strictly speaking that’s NBC (as you mention below, a client/app may 
have been checking for bit-1).

 That said, you hit the relevant point next:



The known flags leaf would only go through backwards compatible changes though 
(since we’d always be additive in adding bits) – assuming bit positions don’t 
change in the protocol.

Exactly.  Unknown became known.  The need to "see" the unknown has gone away.

Thinking on this slightly more, I think I see where our thinking diverges 
somewhat.  From one perspective (illustrated by your point above), bit-1 has 
"gone away".  This is a change.  My perspective is that the semantics of the 
node in question is there to model "unknown".  The change is thus "expected" 
when things are assigned.[>>JTS:] Yeah – I see that perspective. But a client 
using the old API/contract gets new/different behavior for the unknown-flags 
leaf from a new server. Hence NBC – unless we decide in the end to somehow make 
this specific case/typedef an exception but I’m not sure about that yet. The 
typedef and behavior you are describing there may still be useful as-is even if 
we decide to still officially consider the change to unknown-flags behavior as 
NBC (i.e. in new the version of the module that changes a bit from unknown to 
known).

Where things are potentially problematic for end users from a state perspective 
is that if you're testing on the unknown bit, its change is HIGHLY problematic. 
 My use case is for debugging rather than daily operations.
[>>JTS:] OK but when we report NBC vs BC for a new version of a module, we 
can’t really define that based on some use cases and not others. It is based on 
rules purely related to how the API & behavior has changed. This doesn’t make 
the proposed behavior bad – I’m just thinking that we should probably still 
consider it NBC.
 In circumstances where you wanted to always know the contents of the bit 
vector in a constant fashion, a "raw" output is preferable.  However, this 
reverts to the headache discussed in other sub-threads.

I wish to point you and others concerned on these points to the BGP YANG 
modeling for Extended Communities, which will have these problems in a 
different flavor: Known communities will render via the typedefs, unknown will 
render using the prefix 'raw'.  (See typedef bgp-ext-community-type.)  This 
headache is already a consideration in every BGP implementation that deals with 
extended communities in changing meaning.

Re: [netmod] Unknown bits - backwards compatibility

[Jeffrey Haas]

Jason,


> On Apr 12, 2023, at 5:26 PM, Jason Sterne (Nokia)  
> wrote:
> 
> It just occurred to me that to avoid the NBC change we could also consider 
> just calling this new typedef “raw-bits” and always outputting all the bits 
> in the second leaf (whether they are known or not)?  I suspect this was 
> already considered though…

It was and part of the discussion.  As I mentioned to Andy, consistent advice 
for modeling the raw stuff might be helpful.  However, for bit fields I find 
the likely options to be annoying and probably not good ease of use for the 
end-user.

More below

> 
> From: netmod  On Behalf Of Jason Sterne (Nokia)

> One topic that came up during the IETF 116 NETMOD meeting was backwards 
> compatibility.
>  
> From what I understand, a leaf (e.g. unknown-flags) that uses the 
> unknown-bits typedef would never change its definition in YANG. It would 
> always be defined as unknown-bits with all 64 bit definitions even as more 
> and more bits become “known”.  *But* the system would suddenly stop reporting 
> bit-0, then bit-1 in that unknown-flags leaf as those bits become known.

That's the current proposal, yes.

>  
> Strictly speaking, that should probably be considered an NBC change although 
> it is a bit of a grey zone - the NBC rules for state are fuzzy (theoretically 
> they are defined by RFC7950 but it is clear those rules were focused on 
> config, and the same goes for all our new versioning work). But I could 
> imagine an implementation that was previously seeing bit-0 returned and 
> suddenly stops seeing that bit-0 returned (which could cause different 
> interpretation/behavior). So in some ways the semantics of the unknown-flags 
> leaf has changed. It may be better to just flag this as an NBC change so a 
> user would be drawn to look at it, and make a decision that they do or don’t 
> have to update their handling/use of it?

I'm not following the versioning discussions enough to know the full semantics 
you're implying.  That said, you hit the relevant point next:

>  
> The known flags leaf would only go through backwards compatible changes 
> though (since we’d always be additive in adding bits) – assuming bit 
> positions don’t change in the protocol.

Exactly.  Unknown became known.  The need to "see" the unknown has gone away.

Thinking on this slightly more, I think I see where our thinking diverges 
somewhat.  From one perspective (illustrated by your point above), bit-1 has 
"gone away".  This is a change.  My perspective is that the semantics of the 
node in question is there to model "unknown".  The change is thus "expected" 
when things are assigned.

Where things are potentially problematic for end users from a state perspective 
is that if you're testing on the unknown bit, its change is HIGHLY problematic. 
 My use case is for debugging rather than daily operations.  In circumstances 
where you wanted to always know the contents of the bit vector in a constant 
fashion, a "raw" output is preferable.  However, this reverts to the headache 
discussed in other sub-threads.

I wish to point you and others concerned on these points to the BGP YANG 
modeling for Extended Communities, which will have these problems in a 
different flavor: Known communities will render via the typedefs, unknown will 
render using the prefix 'raw'.  (See typedef bgp-ext-community-type.)  This 
headache is already a consideration in every BGP implementation that deals with 
extended communities in changing meaning.

To address that point and needs for stable representation, see leaf-list 
"ext-community-raw".

Netmod's deep scrutiny on this topic is heartily invited. :-)  I expect a lot 
of heartburn over what current implementations do with this stuff.

>  
> It would probably help to show an example of what a YANG module looks like 
> for step 1 and then step 2 (an unknown becomes known), and also what is 
> returned in NETCONF in step 1 and step 2. You partially have that in section 
> 3.2 although the YANG model isn’t shown and it might be good to explicitly 
> mention that  isn’t returned (note it may also be valid to 
> return ).

Figure 4 in draft version -02 shows how the type gets modified to add the new 
bit. The model doesn't change between step 1/ step 2 beyond this.  

If you'd find it helpful, I could add to the text covering Figure 8 "after 
assignment, bit-1 is no longer returned in unknown-flags".  Is that what you're 
looking for?

-- Jeff

___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod

Re: [netmod] Unknown bits - backwards compatibility

[Jason Sterne (Nokia)]

It just occurred to me that to avoid the NBC change we could also consider just 
calling this new typedef "raw-bits" and always outputting all the bits in the 
second leaf (whether they are known or not)?  I suspect this was already 
considered though...

Jason

From: netmod  On Behalf Of Jason Sterne (Nokia)
Sent: Wednesday, April 12, 2023 5:24 PM
To: Jeffrey Haas ; netmod@ietf.org
Subject: [netmod] Unknown bits - backwards compatibility

Hi Jeff,

One topic that came up during the IETF 116 NETMOD meeting was backwards 
compatibility.

>From what I understand, a leaf (e.g. unknown-flags) that uses the unknown-bits 
>typedef would never change its definition in YANG. It would always be defined 
>as unknown-bits with all 64 bit definitions even as more and more bits become 
>"known".  *But* the system would suddenly stop reporting bit-0, then bit-1 in 
>that unknown-flags leaf as those bits become known.

Strictly speaking, that should probably be considered an NBC change although it 
is a bit of a grey zone - the NBC rules for state are fuzzy (theoretically they 
are defined by RFC7950 but it is clear those rules were focused on config, and 
the same goes for all our new versioning work). But I could imagine an 
implementation that was previously seeing bit-0 returned and suddenly stops 
seeing that bit-0 returned (which could cause different 
interpretation/behavior). So in some ways the semantics of the unknown-flags 
leaf has changed. It may be better to just flag this as an NBC change so a user 
would be drawn to look at it, and make a decision that they do or don't have to 
update their handling/use of it?

The known flags leaf would only go through backwards compatible changes though 
(since we'd always be additive in adding bits) - assuming bit positions don't 
change in the protocol.

It would probably help to show an example of what a YANG module looks like for 
step 1 and then step 2 (an unknown becomes known), and also what is returned in 
NETCONF in step 1 and step 2. You partially have that in section 3.2 although 
the YANG model isn't shown and it might be good to explicitly mention that 
 isn't returned (note it may also be valid to return 
).

Jason


From: netmod mailto:netmod-boun...@ietf.org>> On 
Behalf Of Jeffrey Haas
Sent: Monday, April 10, 2023 2:48 PM
To: netmod@ietf.org
Subject: Re: [netmod] Request for WG adoption, 
draft-haas-netmod-unknown-bits-01.txt


CAUTION: This is an external email. Please be very careful when clicking links 
or opening attachments. See http://nok.it/ext for additional information.


Netmod Working Group,

The presentation at IETF 116 on the YANG unknown bits typedef seemed to be 
positively received.

I've updated the draft in version -02 to incorporate a hallway suggestion (from 
Rob, I think?) to rename the bits from a pattern of "unknown-N" to "bit-N".  
Please find the information for the updated draft pasted below my original 
request for adoption.

Having given my presentation and seeing there seems to be interest in the work, 
could we consider adoption?

-- Jeff




On Feb 20, 2023, at 1:23 PM, Jeffrey Haas 
mailto:jh...@pfrc.org>> wrote:

The current version of this small draft has addressed the discussion points to 
date.

I'd like to request that the netmod WG consider adopting this draft.  
Alternatively, if it's thought useful but more appropriate to carry this work 
on elsewhere (e.g. rtgwg), I'd appreciate such feedback.

For the process folk, I am unaware of any applicable IPR covering this draft.

-- Jeff


Name:  draft-haas-netmod-unknown-bits
Revision: 02
Title: Representing Unknown YANG bits in Operational State
Document date: 2023-04-10
Group: Individual Submission
Pages: 18
URL:
https://www.ietf.org/archive/id/draft-haas-netmod-unknown-bits-02.txt
Status: https://datatracker.ietf.org/doc/draft-haas-netmod-unknown-bits/
Html:   
https://www.ietf.org/archive/id/draft-haas-netmod-unknown-bits-02.html
Htmlized:   
https://datatracker.ietf.org/doc/html/draft-haas-netmod-unknown-bits
Diff:   
https://author-tools.ietf.org/iddiff?url2=draft-haas-netmod-unknown-bits-02

Abstract:
  Protocols frequently have fields where the contents are a series of
  bits that have specific meaning.  When modeling operational state for
  such protocols in YANG, the 'bits' YANG built-in type is a natural
  method for modeling such fields.  The YANG 'bits' built-in type is
  best suited when the meaning of a bit assignment is clear.

  When bits that are currently RESERVED or otherwise unassigned by the
  protocol are received, being able to model them is necessary in YANG
  operational models.  This cannot be done using the YANG 'bits' built-
  in type without assigning them a name.  However, YANG versioning
  rules do not permit renaming of named bits.

  This draft proposes a methodology to represent unknown bits in YANG
  operational models and creates a YANG typedef to assist in uniformly