As promised, I've written a proposal for CBOR. See more details and rationale
in https://jira.iotivity.org/browse/IOT-449.
Summary: the proposal is to
* update documentation, variable names and comments that assumed that JSON
would be the only type of payload
* the OIC protocol be modified so that any information that the IoTivity C
stack may need in order to route packets properly be moved from the CoAP
payload to the CoAP options headers (for example, device unique IDs
the C stack be extended with an API for setting and retrieving the payload
type (CoAP Content-Format)
* decouple the encoding of the payload from the data transmission
* remove the dependency on cJSON and remove the sources from the IoTivity
repository
* add an API to the C SDK to allow one to encode and decode CBOR payloads
* modify the C++ stack to use the API from the previous bullet
I will provide the encoder and decoder for CBOR in the form of an extlib that
we can add to IoTivity and replace cJSON with it. Time-permitting, I will also
benchmark it against cJSON.
Expected FAQ:
Q: Will the users of the C++ API be affected?
A: No, the C++ API already abstracted the encoding of the payload in
OCRepresentation, an attribute map class. It previously used Cereal to encode
to JSON. We'll most likely retain Cereal and write a CBOR sink that uses the
library I'm going to create.
Q: Will users of the C API be affected?
A: Yes, but the extent depends on how much they want to change. Right now, our
CoAP stack enforces a Content-Format value of 50 (JSON) and forces users of
the C API to encode and decode JSON on their own. Some people are even writing
hand-written JSON parsers, which is a big problem from the security and
compatibility perspectives.
The "to-be" scenario will allow the user to specify the Content-Format header
option, so users that wish to send JSON will still be able to.
But we will also make a very small CBOR encoder/decoder library available and
we will change the default Content-Format to type 60 (CBOR).
Q: What's this about CoAP Option Headers?
A: Right now, the C stack needs to extract some information from the CoAP
payload in order to properly route the incoming messages. This is especially
true for the device ID, which is right now carried as part of the CoAP payload
(an OIC property).
This proposal moves such properties from the CoAP payload to the CoAP Options
Headers. To illustrate, take the following HTTP-equivalent reply:
200 Ok
Content-type: application/json
{"oc":{"id":"6958df35-af35-46e4-a2fa-f655e938b040","rep":{"state":true}}}
It would become (still with a JSON payload):
200 Ok
Content-type: application/json
X-OIC-Id: 6958df35-af35-46e4-a2fa-f655e938b040
{"oc":{"rep":{"state":true}}}
This way, the C stack need not depend on the encoding type for its routing
needs. Everything it needs to make decisions is in the CoAP headers or HTTP
headers.
Q: How about protocols that don't have option headers?
A: For those, the IoTivity stack needs should come up with its own headers and
separate the user (service) payload. For example, it can write a 16-bit value
indicating the header length and encode the information there.
Using CBOR for this is a good idea, but not required. The user payload would
be carried in binary form, so no transformation would be required to pass it
to upper layers of the stack.
Q: How much smaller is CBOR compared to JSON?
A: It depends on the data being transmitted. CBOR does not compress strings
like zlib, so any strings that exist in JSON would be carried exactly as-is
(we're not using any escape sequences in IoTivity right now).
Numbers, booleans and the overhead for maps and arrays are much smaller.
The following is an example from the spec:
{"oc":[{"href":"/a/room"},{"href":"/a/light","rep":
{"state":"false","color":"0"}},{"href":"/a/fan","rep":
{"state":"true","speed":10}}]}
If we correct the silly mistakes of sending numbers and booleans as strings,
the JSON payload is 129 bytes (too big for 6LoWPAN without fragmentation). The
equivalent CBOR encoding is 85 bytes, which would fit a 6LoWPAN frame even in
the worst case scenario.
Q: What would the API look like?
A: Something like this:
CBORValue value;
if (!cbor_map_get_value(&request, "href", &value) ||
!cbor_string_equals(&value, "/a/light"))
return;
CBORValue rep;
if (!cbor_map_get_value(&request, "rep", &rep) ||
!cbor_value_type(&map) == CBOR_TYPE_MAP)
return;
// extract properties:
bool state;
char *color;
if (cbor_map_get_value(&map, "state", &value)
&& cbor_value_get_boolean(&value, &state))
set_light_state(state);
if (cbor_map_get_value(&map, "color", &value)
&& cbor_value_get_string(&value, &color)) {
set_light_color(color);
free(color);
}
As you can see, all string comparisons except for the extraction of the colour
are zero-copy (in situ).
Q: Why not EXI?
A: Because we're CBOR is designed to be compatible with JSON, so the
conversion is much easier. EXI is designed to be compatible with XML.
Q: Why not MsgPack? Why not CapnProto? Why not UBIJSON? Why not ...?
A: Short answer: because Pat told me to use CBOR.
Long answer: each of those has its advantages but also disadvantages. CBOR
wins against them because:
* it has its own CoAP Content-Format registry number (60), so we don't need
to write "application/cbor"
* it's very space-efficient, trading off some decoding speed for it
* it's extensible, unlike MsgPack
--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
