Hi folks,
As a few of us were working on modeling the ALTO protocol using YANG, we
were pondering on a more general question: can YANG model all JSON based
protocols? What is the condition for a JSON based protocol (at least the
message format) to have an syntactically equivalent, hence interoperable,
YANG model with JSON encoding? Alternatively, in order to interoperate,
semantic equivalence might be sufficient, is there any condition for
semantic equivalence?
tl;dr: A JSON based protocol carried by HTTP can have syntactically
equivalent YANG model, if and only if all the keys in the key-value pairs
in the JSON message are pre-defined keywords in the protocol.
We've come up with a few ideas, and a few things below are work in
progress, but we would love your feedback!
Thank you!
Xiao
======================
1. Introduction.
JavaScript Object Notation (JSON) has been a popular choice as the message
encoding for many network protocols such as the Application Layer Traffic
Optimization (ALTO) protocol, the Content Delivery Networks Interconnection
(CDNi) protocol, etc.
Meanwhile, there are broad interests in the networking community to use
YANG to define data model so that one can use YANG related tools such as
OpenDayLight controller. Although YANG itself is XML based, there have been
efforts to model JSON content using YANG [draft-ietf-netmod-yang-json-01].
A natural question rises: can YANG model all JSON based protocols? What is
the condition for a JSON based protocol (at least the message format) to
have an syntactically equivalent, hence interoperable, YANG model with JSON
encoding? Alternatively, in order to interoperate, semantic equivalence
might be sufficient, is there any condition for semantic equivalence?
2. Claim
A JSON based protocol carried by HTTP can have syntactically equivalent
YANG model, if and only if:
(1) the message encoding condition is met;
(2) the uri condition is met.
2.1. The message encoding condition
The JSON message encoding MUST not contain a variable as a key in a JSON
object key-value pair. In other words, the keys in the JSON message must be
an already defined constant (or keyword) in the message format of the
protocol.
For example, if I have a protocol where “network-map”, “src”, and “dsts”
are defined keywords, JSON text A meets the condition whereas B does not
because “PID1” and “PID2” are not protocol keywords (they are resource IDs).
A:
{
“network-map”: [
{
“src”: “PID1”,
“dsts”: [“PID2”, “PID3”]
},
{
“src”: “PID2”,
“dsts”: [“PID3”, “PID4”]
}
]
}
B:
{
“network-map”: [
“PID1: [“PID2”, “PID3”],
“PID2”: [“PID3”, “PID4”]
]
}
>From this we know that since ALTO protocol uses encoding B, there cannot be
a syntactically equivalent YANG model.
2.2 The URI condition
Some of the YANG-related protocols might have URI constraints, e.g.
RESTCONF. For now, we assume either that the JSON-based protocol URI could
be conformed to RESTCONF compliant uri, or that the server could have a
routing mapping between the protocol compliant uri and the RESTCONF
compliant uri, hence this condition would not be an issue, which allows us
to focus on the message encoding condition.
3. Proof
3.1. The message encoding condition is necessary.
We first note that this condition is a necessary condition for a JSON based
protocol to have a syntactically equivalent YANG model by proving its
contrapositive.
If one of the keys in the key-value pair in the JSON document is not
pre-defined, the corresponding XML tags will not be pre-defined keywords.
Therefore, it would not possible to model it in YANG without using
YANG’s anyxml
statement (which allows arbitrary XML content).
However, using the anyxml statement would defeat our purpose of modeling
the data as it allows arbitrary XML content, and will not be helpful in the
subsequent parsing process.
3.2. The message encoding condition is sufficient.
We prove this by providing a translation procedure from a JSON message that
is compliant with the protocol we are trying to model, to a custom java
class that can be used for Jackson data binding, then to a YANG model.
We note that the middle step of translating to and from the custom parser
class is not necessary, but it will be useful.
3.2.1. motivation
JSON data binding is the process of binding data structures (objects,
arrays, etc.) in JSON to the appropriate data structures in the server
(e.g. java classes, database tables, etc.) in parsing the JSON text. In
order to process JSON messages in a meaningful manner, data binding is
necessary. Even if the binding is not explicit, the server would need to do
it eventually. For example, one can read JSON content in a stream without
binding it to the java classes, but eventually in order to make sense of
the data, the server would eventually have to organize it, which is roughly
analogous to data binding upfront. Popular choices for JSON parsing and
data binding include jackson and gson.
We use Jackson full data binding as our approach. Full data binding binds
JSON content into plain old java objects (POJOs), i.e. this custom parser
class can neither extend nor implement any other class. Jackson uses
ObjectMapper with the custom parser class to parse JSON content into this
class.
3.2.2. The message encoding condition
The message encoding condition is that all keys in each key-value pair in
the JSON text must be pre-defined keywords. As the keys will become either
class names and instance variable names, or be keys in the java maps, it is
easy to see that the condition is equivalent to “there exists a full data
binding in Jackson custom parser class without using any map structures
(Map<String, ?>).”
3.2.3. Proof
We provide a recursive binding process from a JSON object to the Jackson
custom parser class to be used by Jackson ObjectMapper.
Type determine_type(value) {
if (value is of a primitive type, i.e. string, number, boolean, null) {
return the corresponding java primitive type;
}
if (value is a JSON object) {
return build_parser_class(value).class;
}
if (value is an array) {
return ArrayList<T> where T=determine_type(value[0]);
}
// should not reach here.
}
Class build_parser_class(JSONObject obj) {
create custom class C;
for each key/value pair in the obj {
add instance variable v in C;
the name of variable v <- key; // (__known__ a/c to our assumption)
the type of variable v <- determine_type(value);
}
return C.class;
}
Naming:
--change everything into CamelCase (i.e. remove dashes, etc.)
--for instance variables, use “my” prefix, (e.g. myVariable, myNetworkMap,
etc.)
--for the custom class name, if the object is an element of the array, use
“Element” suffix.
This is just one convention so that the next step proceeds smoothly. As
long as this naming translation is consistent with the naming stage in the
next step, it will work just fine.
Example (a modified ALTO protocol network map example):
JSON object:
{
“meta”: {
“resource-id”: “my-default-map”,
“tag”: “aab875ef69c87d012”
},
“network-map”: [
{
“src”: “PID1”,
“dsts”: [“PID1”, “PID2”, “PID3”]
},
{
“src”: “PID2”,
“dsts”: [“PID1”, “PID3”]
},
{
“src”: “PID3”,
“dsts”: [“PID2”, “PID3”]
}
]
}
Result of build_parser_class(obj):
Class JSONObject {
Meta myMeta;
ArrayList<NetworkMapElement> myNetworkMap;
}
Class Meta {
String myResourceId;
String myTag;
}
Class NetworkMapElement {
String mySrc;
ArrayList<String> myDsts;
}
Now given the Jackson Parser Java Class, to get a syntactically equivalent
YANG model:
YANGModel build_yang_model(Class C) {
for each instance variable (Type, Name) {
if (Type is primitive type: string, number, boolean, null) {
add the following to the YANG module:
“leaf Name { type <YANG equivalent of Type>; }”
}
if (Type is an ArrayList<TypeElement>) {
if (TypeElement is primitive type) {
add the following to the YANG module:
“leaf-list Name { type <YANG equivalent of TypeElement>; }”
} else {
// TypeElement is a custom parser class
add the following to the YANG module:
“list Name { <result from build_yang_model<TypeElement.class>> }”
}
}
if (Type is a custom parser class) {
add the following to the YANG module:
“container Name { <result from build_yang_model<Type.class>> }”
}
}
}
Result from the previous example:
container meta {
leaf resource-id {
type string;
}
leaf tag {
type string;
}
}
list network-map {
leaf src {
type string;
}
leaf-list dsts {
type string;
}
}
This does validate the JSON document with XML-JSON encoding. For your
reference, this is the XML document which validates:
<?xml version="1.0" encoding="UTF-8" ?>
<meta>
<resource-id>my-default-map</resource-id>
<tag>aab875ef69c87d012</tag>
</meta>
<network-map>
<src>PID1</src>
<dsts>PID1</dsts>
<dsts>PID2</dsts>
<dsts>PID3</dsts>
</network-map>
<network-map>
<src>PID2</src>
<dsts>PID1</dsts>
<dsts>PID3</dsts>
</network-map>
<network-map>
<src>PID3</src>
<dsts>PID2</dsts>
<dsts>PID3</dsts>
</network-map>
This proves that the message encoding condition is a sufficient condition
for the JSON object to have a YANG model.
Note the model generated is very crude and lose almost all constraints and
all inheritance features (if any), because it focuses on the syntax and is
essentially converted from an JSON object compliant with a protocol instead
of from the protocol itself. Hence this result is more useful in
determining which JSON based protocols cannot have a syntactically
equivalent YANG model, than in generating a good YANG model.
3.3. Conclusion
Our claim holds. A JSON based protocol carried by HTTP can have
syntactically equivalent YANG model, if and only if all the keys in the
key-value pairs in the JSON message are pre-defined keywords.
4. Semantic equivalence
For JSON based protocols that don’t satisfy the message encoding condition,
it is still possible to have a semantically equivalent YANG model. All that
is required for the protocol compliant clients and the YANG model compliant
server to interoperate is an adapter which does the following:
1) translate FROM YANG server compliant response msg TO alto compliant
response msg
2) translate FROM alto compliant request msg TO YANG server compliant
request msg
4.1. Claim
This adapter needs to be protocol-aware.
Ideally, given any YANG model, we would like to be able to automatically
(or at least mechanically) generate this message adapter, which means not
looking at the protocol or its compliant msgs. However, without knowing the
specific protocol that we are working with (i.e. human intervention, i.e.
looking at the protocol compliant msgs), such an adapter cannot be
auto-generated.
4.2. Proof by Indistinguishability
Suppose both the YANG server compliant msg m_y and the actually protocol
compliant msg m_p are in JSON (or have been encoded into JSON). Looking at
the differences between the two messages, call these differences {d1, d2,
..., dn}. The goal for the auto-generated adapter would be to identify and
eliminate these differences. Construct a new JSON msg m' where all but one
difference di is the same as m_p and di is the same as the m_y. Without
looking at the protocol (or m_p), the auto-generated adapter would not be
able to distinguish between m' and m_p in its translation process, which
means, it won't be able to tell whether it should change di or not. Hence,
such an adapter must be protocol-aware.
A good example is the dependent-vtag in the ALTO protocol:
"dependent-vtag" : [
{
"resource-id" : "my-network-map",
"tag" : "abcd1234"
}
]
It was specified this way in the alto protocol. However, it could
conceivably be the case that it was originally the following map structure,
and was converted into the above encoding because of the map->list+key
issue. (This case is actually one of the few differences in the m_y and m_p
where the adapter does not need to convert it back to a map structure.)
"dependent-vtag" : {
"my-network-map" : {
"tag" : "abcd1234"
}
}
Without knowing the protocol, there is no way to tell.
5. Ramifications
We now understand the basic condition for a JSON based protocol to have a
YANG Model. For the protocols that don’t meet this condition, there can be
a semantic equivalent YANG model, but there won’t be a generic process of
generating the adapter for all such protocols.
_______________________________________________
alto mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/alto
