Re: [netmod] json empty leaf encoding strangeness [Re: WG Last Call for draft-ietf-netmod-yang-json-04 (until 2015-06-29)]
On Mon, Dec 20, 2021 at 1:53 PM Christian Hopps wrote: > > > > On Dec 18, 2021, at 11:33 AM, Andy Bierman wrote: > > > > Thank you for caring about backward compatibility and stability. > > > > I guess I left as implicit in my question of whether we could do something > to fix it, that that "could" implied a fix wouldn't break things. I > certainly care about stability and backward compatibility. > > good point. Sometimes we forget that changing the RFCs is the easy part. People that don't even know this mailing list exists have to learn and implement the changes. Then the operator community has to deal with tool incompatibilities for years to come. The "industry" expects stability from YANG at this point. They are focused on implementing and deploying YANG models. The foundation is good enough. Not perfect. Nobody is expecting that. Thanks, > Chris. Andy ___ netmod mailing list netmod@ietf.org https://www.ietf.org/mailman/listinfo/netmod
Re: [netmod] json empty leaf encoding strangeness [Re: WG Last Call for draft-ietf-netmod-yang-json-04 (until 2015-06-29)]
> On Dec 18, 2021, at 11:33 AM, Andy Bierman wrote: > > Thank you for caring about backward compatibility and stability. > I guess I left as implicit in my question of whether we could do something to fix it, that that "could" implied a fix wouldn't break things. I certainly care about stability and backward compatibility. Thanks, Chris. ___ netmod mailing list netmod@ietf.org https://www.ietf.org/mailman/listinfo/netmod
Re: [netmod] json empty leaf encoding strangeness [Re: WG Last Call for draft-ietf-netmod-yang-json-04 (until 2015-06-29)]
Carsten Bormann writes: > On 18. Dec 2021, at 19:04, Ladislav Lhotka wrote: >> >>'foo: null'? Doesn't that make testing for empty values a major >>pain? 'if (foo)' would not work. > > Same with »false«, which 7951 is not escaping into an array. This > argument simply doesn’t hold water. As was mentioned, you would > check »”foo” in o«, which works for null, false, and all other > values that may happen to be falsy(*) in the platform. The whole story is that Phil wanted to have both empty and boolean values as JSON strings. The result was a compromise, and I thought that [null] was a much better choice because, unlike "null" string, it cannot otherwise appear as a valid value of a YANG leaf. > >>JSON evolved from Javascript, so it must keep the javascript >>meanings for these keywords. > > That is a common misconception that must be stamped out, but we have > j...@ietf.org for that discussion. It's not that I am defending these views. I did argue with RFC 4627, but not too hard, as this seemed like a marginal issue at that time - much more fundamental objections against the JSON representation were raised back then. > >> It is indeed true that tests in JavaScript cannot really distinguish between >> a non-existent member and member with the value of 'null'. I remember I >> wasn't happy about this change but I thought it wasn't a big deal. > > As mentioned, they can. > And, yes, this little unnecessary complexity is not a big deal even if it is > based on mistaken beliefs; every non-trivial protocol has some unburied > zombies in the basement. Right, in hindsight it was a mistake. It is also worth noting that [null] is only an on-the-wire value, so e.g. None can be used internally in Python. I have written a lot of related running code and don't find it a serious problem. Lada > > Grüße, Carsten > > (*) falsy: converted to false when implicitly coerced to a Boolean (Ant.: > truthy) > -- Ladislav Lhotka Head, CZ.NIC Labs PGP Key ID: 0xB8F92B08A9F76C67 ___ netmod mailing list netmod@ietf.org https://www.ietf.org/mailman/listinfo/netmod
Re: [netmod] json empty leaf encoding strangeness [Re: WG Last Call for draft-ietf-netmod-yang-json-04 (until 2015-06-29)]
On 18. Dec 2021, at 19:04, Ladislav Lhotka wrote: > >'foo: null'? Doesn't that make testing for empty values a major >pain? 'if (foo)' would not work. Same with »false«, which 7951 is not escaping into an array. This argument simply doesn’t hold water. As was mentioned, you would check »”foo” in o«, which works for null, false, and all other values that may happen to be falsy(*) in the platform. >JSON evolved from Javascript, so it must keep the javascript >meanings for these keywords. That is a common misconception that must be stamped out, but we have j...@ietf.org for that discussion. > It is indeed true that tests in JavaScript cannot really distinguish between > a non-existent member and member with the value of 'null'. I remember I > wasn't happy about this change but I thought it wasn't a big deal. As mentioned, they can. And, yes, this little unnecessary complexity is not a big deal even if it is based on mistaken beliefs; every non-trivial protocol has some unburied zombies in the basement. Grüße, Carsten (*) falsy: converted to false when implicitly coerced to a Boolean (Ant.: truthy) ___ netmod mailing list netmod@ietf.org https://www.ietf.org/mailman/listinfo/netmod
Re: [netmod] json empty leaf encoding strangeness [Re: WG Last Call for draft-ietf-netmod-yang-json-04 (until 2015-06-29)]
Christian Hopps writes: >> On Dec 18, 2021, at 7:59 AM, Carsten Bormann wrote: >> >> On 2021-12-18, at 09:20, Eliot Lear wrote: >>> >>> Still it seems like the right thing to do, in order to fall into line with >>> the target language spec. >> >> But that’s not what YANG was designed for. >> YANG is a data modeling language, which incidentally has a couple of >> representation formats (YANG-XML, YANG-JSON, YANG-CBOR). >> There is no intention for the YANG generic data model (the set of all data >> models YANG can express) to cover the generic data model of the >> representation format. > > To be clear, the question I raised is about an encoding of YANG modeled data. > In particular the JSON encoding of the YANG empty leaf element. > > https://datatracker.ietf.org/doc/html/rfc7951#section-6.9 > > the justification for having the very odd encoding was that there was some > belief that some programming languages might have trouble with the obvious > JSON "null" encoding (which stands for "no value" in JSON -- exactly what we > want here). An old email gave a python example to try and illustrate this, > except the python example was an incorrect use of python for this case > (presence check) and not a problem with the natural JSON encoding for no > value items. In fact correctly written python code has no problem at all with > the more natural "null" encoding, and results in the correct internal > representation in python to boot. > > This isn't about the YANG specification in general, or making YANG > conform to any particular language. It's just about the a choice in > the specification for the encoding of YANG in a particular format > (JSON). The original proposal (even before my individual draft appeared in the NETMOD WG) was actually just "null". We'd been discussing it back in March 2012 in a smaller group (Andy, Jürgen, Martin, Phil Shafer and myself), and Phil objected to this encoding of empty values. His arguments were, for example: 'foo: null'? Doesn't that make testing for empty values a major pain? 'if (foo)' would not work. JSON evolved from Javascript, so it must keep the javascript meanings for these keywords. It is indeed true that tests in JavaScript cannot really distinguish between a non-existent member and member with the value of 'null'. I remember I wasn't happy about this change but I thought it wasn't a big deal. Lada > > Thanks, > Chris. > > > >> >> While RFC 8791 (and before it RFC 8040) extended YANG to be able to describe >> data in flight, there is no intention that YANG can be used to describe the >> entire expressibility of the representation format. We have Relax-NG, CDDL, >> and a few other modeling approaches if we need that. > >> Grüße, Carsten >> > -- Ladislav Lhotka Head, CZ.NIC Labs PGP Key ID: 0xB8F92B08A9F76C67 ___ netmod mailing list netmod@ietf.org https://www.ietf.org/mailman/listinfo/netmod
Re: [netmod] json empty leaf encoding strangeness [Re: WG Last Call for draft-ietf-netmod-yang-json-04 (until 2015-06-29)]
On Sat, Dec 18, 2021 at 6:14 AM Carsten Bormann wrote: > On 2021-12-18, at 14:35, Christian Hopps wrote: > > > > This isn't about the YANG specification in general, or making YANG > conform to any particular language. It's just about the a choice in the > specification for the encoding of YANG in a particular format (JSON). > > I understand the objective to have a good representation for YANG data. > > For instance, in YANG-CBOR, a YANG empty is represented as a CBOR null > (0xf6, in case you need to visualize the bits); there is no expectation > that this will be difficult to handle by CBOR implementations. > > However, there is nothing wrong with representing empty with the array > »[null]« in YANG-JSON. It is just more complex, and apparently > unnecessarily so. > Whether removing this complexity to save two characters is worth doing an > incompatible upgrade to the YANG-JSON representation format can be debated. > I’d say: emphatically no. > > Thank you for caring about backward compatibility and stability. The other motivation that came up in this thread is that it is natural to > have null as a value in JSON and YANG cannot describe such formats via > YANG-JSON. This was what I was reacting to, but, as you say, that wasn’t > your question. > > Grüße, Carsten > > Andy > ___ > netmod mailing list > netmod@ietf.org > https://www.ietf.org/mailman/listinfo/netmod > ___ netmod mailing list netmod@ietf.org https://www.ietf.org/mailman/listinfo/netmod
Re: [netmod] json empty leaf encoding strangeness [Re: WG Last Call for draft-ietf-netmod-yang-json-04 (until 2015-06-29)]
> On Dec 18, 2021, at 7:59 AM, Carsten Bormann wrote: > > On 2021-12-18, at 09:20, Eliot Lear wrote: >> >> Still it seems like the right thing to do, in order to fall into line with >> the target language spec. > > But that’s not what YANG was designed for. > YANG is a data modeling language, which incidentally has a couple of > representation formats (YANG-XML, YANG-JSON, YANG-CBOR). > There is no intention for the YANG generic data model (the set of all data > models YANG can express) to cover the generic data model of the > representation format. To be clear, the question I raised is about an encoding of YANG modeled data. In particular the JSON encoding of the YANG empty leaf element. https://datatracker.ietf.org/doc/html/rfc7951#section-6.9 the justification for having the very odd encoding was that there was some belief that some programming languages might have trouble with the obvious JSON "null" encoding (which stands for "no value" in JSON -- exactly what we want here). An old email gave a python example to try and illustrate this, except the python example was an incorrect use of python for this case (presence check) and not a problem with the natural JSON encoding for no value items. In fact correctly written python code has no problem at all with the more natural "null" encoding, and results in the correct internal representation in python to boot. This isn't about the YANG specification in general, or making YANG conform to any particular language. It's just about the a choice in the specification for the encoding of YANG in a particular format (JSON). Thanks, Chris. > > While RFC 8791 (and before it RFC 8040) extended YANG to be able to describe > data in flight, there is no intention that YANG can be used to describe the > entire expressibility of the representation format. We have Relax-NG, CDDL, > and a few other modeling approaches if we need that. > Grüße, Carsten > ___ netmod mailing list netmod@ietf.org https://www.ietf.org/mailman/listinfo/netmod
Re: [netmod] json empty leaf encoding strangeness [Re: WG Last Call for draft-ietf-netmod-yang-json-04 (until 2015-06-29)]
On 2021-12-18, at 09:20, Eliot Lear wrote: > > Still it seems like the right thing to do, in order to fall into line with > the target language spec. But that’s not what YANG was designed for. YANG is a data modeling language, which incidentally has a couple of representation formats (YANG-XML, YANG-JSON, YANG-CBOR). There is no intention for the YANG generic data model (the set of all data models YANG can express) to cover the generic data model of the representation format. While RFC 8791 (and before it RFC 8040) extended YANG to be able to describe data in flight, there is no intention that YANG can be used to describe the entire expressibility of the representation format. We have Relax-NG, CDDL, and a few other modeling approaches if we need that. Grüße, Carsten ___ netmod mailing list netmod@ietf.org https://www.ietf.org/mailman/listinfo/netmod
Re: [netmod] json empty leaf encoding strangeness [Re: WG Last Call for draft-ietf-netmod-yang-json-04 (until 2015-06-29)]
Just following this up, I see in my own work it occurs a little bit, so
it is out there. It doesn't create ambiguities, tho, because it is used
in null-valued leafs. It just creates syntax breakage. Some parsers
will have a problem with recognizing a naked null as valid. On the
other hand, in the case of MUD, we separately added versioning into the
model, so we could bump this if the serialization updates.
On 18.12.21 09:20, Eliot Lear wrote:
Possibly. The issue with fixing it is that it would create an
ambiguity between null and [ null ]. I am not sure how often [ null
] would not have that semantic equivalence in earlier work. This is
further complicated by a lack of self-descriptive versioning in YANG
serializations.
Still it seems like the right thing to do, in order to fall into line
with the target language spec.
Eliot
On 18.12.21 07:47, Christian Hopps wrote:
Hi,
I was just going over json encoding of YANG, and got to the empty
leaf encoding and it's odd use of "[null]" rather than just "null". I
read the explanation, but had a hard time believing what it said. So,
running code time. The first thing I tried was Python to see if
something weird happened:
In [1]: import json
In [2]: json.loads('{"foo": null}')
Out[2]: {'foo': None}
Nope, seems fine. Then checked the "definitive source" for json,
javascript:
js> JSON.parse("{\"foo\": 1, \"bar\": null}")
({foo:1, bar:null})
Seems fine here too..
So I went searching the mailing list and found this:
On Jun 29, 2015, at 5:49 AM, Ladislav Lhotka wrote:
Juergen Schoenwaelder writes:
...
- I am not sure I understand the argument why [null] is better than
null for the empty type. Perhaps it is but the text does not
really
tell me.
"foo": null
may sometimes appear as if the "foo" instance is not present at
all - in
some languages this is probably more serious than in others.
This is how
it works in Python:
import json
obj = json.loads('{"foo": null, "bar": [null]}')
"YES" if obj["foo"] else "NO"
'NO'
"YES" if obj["bar"] else "NO"
'YES'
So the special value "[null]" seems safer and it cannot appear
in any
other context.
The justifying example here is, unfortunately, wrong. One does not
(normally) check for the presence of a dictionary item in python by
trying to access the value of that item (which is what the code above
is doing). The standard python code to check for the presence of an
item is as follows, and it works fine with a "null" json value:
In [1]: import json
In [2]: o = json.loads('{"foo": null}')
In [3]: o
Out[3]: {'foo': None}
In [4]: "YES" if "foo" in o else "NO"
Out[4]: 'YES'
Indeed in python if you try and check for a missing item the way the
email above describes, and the item is missing, an exception is raised:
In [5]: o["bar"]
---
KeyError Traceback (most recent
call last)
One *could* catch the "KeyError" to check for missing items which
will also work using "null" son values for empty leafs, but the more
common form is "if key in dict" in my experience.
Furthermore, using "access the value" for presence check will also
"fail" for zero values and for empty strings, but we don't have odd
encodings for integers or strings...
In [1]: import json
In [2]: o = json.loads('{"foo": 0, "bar": ""}')
In [3]: o
Out[3]: {'foo': 0, 'bar': ''}
In [4]: "YES" if o["foo"] else "NO"
Out[4]: 'NO'
In [5]: "YES" if o["bar"] else "NO"
Out[5]: 'NO'
Point being, evaluating an items value is not the right way to check
for "presence", it's the way to check the "value" of the item. An
empty leaf has no value so it's basically never correct to try to
access that value in code.
Javascript is basically the same as python here:
js> o = JSON.parse('{"foo": null}')
({foo:null})
js> if ("foo" in o) { "YES" } else { "NO" }
"YES"
So, do we think it would it ever be possible to change (fix) this
encoding oddity?
Thanks,
Chris.
___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod
___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod
OpenPGP_signature
Description: OpenPGP digital signature
___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod
Re: [netmod] json empty leaf encoding strangeness [Re: WG Last Call for draft-ietf-netmod-yang-json-04 (until 2015-06-29)]
Possibly. The issue with fixing it is that it would create an ambiguity
between null and [ null ]. I am not sure how often [ null ] would not
have that semantic equivalence in earlier work. This is further
complicated by a lack of self-descriptive versioning in YANG serializations.
Still it seems like the right thing to do, in order to fall into line
with the target language spec.
Eliot
On 18.12.21 07:47, Christian Hopps wrote:
Hi,
I was just going over json encoding of YANG, and got to the empty leaf
encoding and it's odd use of "[null]" rather than just "null". I read
the explanation, but had a hard time believing what it said. So,
running code time. The first thing I tried was Python to see if
something weird happened:
In [1]: import json
In [2]: json.loads('{"foo": null}')
Out[2]: {'foo': None}
Nope, seems fine. Then checked the "definitive source" for json,
javascript:
js> JSON.parse("{\"foo\": 1, \"bar\": null}")
({foo:1, bar:null})
Seems fine here too..
So I went searching the mailing list and found this:
On Jun 29, 2015, at 5:49 AM, Ladislav Lhotka wrote:
Juergen Schoenwaelder writes:
...
- I am not sure I understand the argument why [null] is better than
null for the empty type. Perhaps it is but the text does not really
tell me.
"foo": null
may sometimes appear as if the "foo" instance is not present at
all - in
some languages this is probably more serious than in others. This
is how
it works in Python:
import json
obj = json.loads('{"foo": null, "bar": [null]}')
"YES" if obj["foo"] else "NO"
'NO'
"YES" if obj["bar"] else "NO"
'YES'
So the special value "[null]" seems safer and it cannot appear in any
other context.
The justifying example here is, unfortunately, wrong. One does not
(normally) check for the presence of a dictionary item in python by
trying to access the value of that item (which is what the code above
is doing). The standard python code to check for the presence of an
item is as follows, and it works fine with a "null" json value:
In [1]: import json
In [2]: o = json.loads('{"foo": null}')
In [3]: o
Out[3]: {'foo': None}
In [4]: "YES" if "foo" in o else "NO"
Out[4]: 'YES'
Indeed in python if you try and check for a missing item the way the
email above describes, and the item is missing, an exception is raised:
In [5]: o["bar"]
---
KeyError Traceback (most recent call
last)
One *could* catch the "KeyError" to check for missing items which will
also work using "null" son values for empty leafs, but the more common
form is "if key in dict" in my experience.
Furthermore, using "access the value" for presence check will also
"fail" for zero values and for empty strings, but we don't have odd
encodings for integers or strings...
In [1]: import json
In [2]: o = json.loads('{"foo": 0, "bar": ""}')
In [3]: o
Out[3]: {'foo': 0, 'bar': ''}
In [4]: "YES" if o["foo"] else "NO"
Out[4]: 'NO'
In [5]: "YES" if o["bar"] else "NO"
Out[5]: 'NO'
Point being, evaluating an items value is not the right way to check
for "presence", it's the way to check the "value" of the item. An
empty leaf has no value so it's basically never correct to try to
access that value in code.
Javascript is basically the same as python here:
js> o = JSON.parse('{"foo": null}')
({foo:null})
js> if ("foo" in o) { "YES" } else { "NO" }
"YES"
So, do we think it would it ever be possible to change (fix) this
encoding oddity?
Thanks,
Chris.
___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod
OpenPGP_signature
Description: OpenPGP digital signature
___
netmod mailing list
netmod@ietf.org
https://www.ietf.org/mailman/listinfo/netmod
