This is more of a development issue than a main mailing list issue. If
we want to continue past this, it would be good to switch to the
development list.
What I changed is more fundamental than that. I essentially changed how
a protocol version number would be stored...
All layers above this are designed to not need version numbering. But
the lowest level had a stupid oversight in it, and I really don't expect
it to be reasonable to modify this lowest layer.
The basic format of everything we send is type/length/value (TLV). TLV
message formats are quite flexible with respect to growth and extension
without version numbering. The LLDP spec is a good example of how one
can extend it significantly.
Unfortunately, because the packet format is UDP-based, I originally
limited the length field to 2 bytes. HOWEVER, with compression, an
uncompressed field can be above 64K bytes. So this was a brain fart - a
disconnect between the fact that this is UDP-based and the fact that I
had planned for compression - thereby supporting things larger than 64K
bytes. Duhhh!!
Most of the interesting data is sent in JSON - and the JSON is very
flexible in terms of ignoring things you don't understand. This is
where most interesting versioning will occur.
In addition, a single number for protocol versioning is likely
insufficient - since you may need to know if a particular nanoprobe
supports a particular directive. That's not so much a protocol change
as it is an application change/version. And there will be a lot more
than 256 of them... And in open source, version numbers are not easy to
track, and are often not even linear...
Should it become necessary in the future, a more fruitful approach would
likely be to have each nanoprobe return a list of directives it supports
when it first starts up. That would cover the peculiarities of the open
source fork/merge model better than a linear version number. That
combined with the basic flexibility of a TLV protocol combined with JSON
for more complex things would cover all the kinds of oddities one is
likely to run into.
This addition would not require any kind of change format on the wire.
-------------------------------- low-level-format details
--------------------------------------------
General packet format:
Frameset type: 2 bytes
Frameset length: 3 bytes
Frameset flags: 2 bytes
[A UDP packet on the wire is permitted to contain more than one Frameset]
The Frameset header is followed by a series of Frames:
Each Frame is a TLV triple:
Frame Type: 2 bytes
Frame length: 3 bytes
Frame value ("n" bytes where "n" is given by the Frame Length above)
The last Frame in a Frameset is supposed to be type 0, length 0.
The main reason I can think of for updating this low-level format is one of:
16777216 bytes isn't enough for a single uncompressed primitive value
65536 isn't enough distinct message types
Given that the wire protocol is UDP, 2^24 seems like enough bytes. If
we need more than 2^16 basic message types, then we've done something
wrong...
If someone knows of a reason why one would need more than 2^24 bytes for
a single data item ("Frame"), then that would be good to know.
Of course, if that comes up, one could always make a frame type that
says "multi-part frame" followed by "multi-part-frame-continuation" as a
kludge for this kind of frame - and the frame contents then encapsulated
and split across these frames. Ditto for huge framesets...
Since this low-level format is perfectly happy to support binary, this
should work fine (assuming it comes up).
On 01/22/2014 03:00 PM, R P Herrold wrote:
> On Wed, 22 Jan 2014, Alan Robertson wrote:
>
>> Hopefully this won't occur again any time in the foreseeable future...
> the conventional way the RFC's do this, so as to not have to
> rely on hope is to have a protocol version indicator (usually
> not more than 8 bits wide. pPssibly less if it can be
> 'over-loaded' by bit-mapping onto a small range value) in the
> headers, so that one might be strict in what one states, and
> liberal in what one receives
>
> -- Russ herrold
--
Alan Robertson <[email protected]> - @OSSAlanR
"Openness is the foundation and preservative of friendship... Let me claim
from you at all times your undisguised opinions." - William Wilberforce
_______________________________________________
Assimilation mailing list - Discovery-Driven Monitoring
[email protected]
http://lists.community.tummy.com/cgi-bin/mailman/listinfo/assimilation
http://assimmon.org/
