Hi Peter,
> Le 4 juin 2015 à 11:28, peter van der Stok <[email protected]> a écrit :
>
> Hi Alexander,
>
> Your suggestion to standardize a handle to a mechanism to do hash aliasing
> but keep the algorithms out of CoMI seems reasonable to me in a first
> approach.
> It is then a matter of interest whether the mechanisms of hash aliasing are
> standardized within CoRE wg or even within CoMI I-D.
>
> However, I think the idea is interesting and like to investigate it further;
> and understand what extra standardization is needed.
If the YANG hash compression + reservation of < 256 hash values is accepted to
CoMI, which I believe can be achieved with minimal modification and at no cost
in terms of implementation or resource requirements, there are two main
approaches to implementing the hash aliases:
1) Independent of the YANG schema
1.a) From a URI, which can be local (e.g /mg/yang.hash-alias or including the
syntax in the future /mg/yang.hash with special provisioning of having aliased)
or remote
1.b) Reserve one of the alias values for the alias mapping resource (e.g.
/mg/0)
2) Dependent on the YANG schema
2.a) By using a leaf in the schema (e.g. alternate_hash = 1, or
comi:alternate_hash = 3)
2.b) By using an URI specified in the schema (not sure if this would be very
useful though)
I would say that it should be possible to have the aliasing independent on the
schema, as a given operator may decide on a different operation modes for the
same YANG schema.
==========================
1.a)
If there is a dedicated file (e.g. /mg/yang.hash-alias), it could be updated by
the client if necessary. It would be nice to have a way to make sure that the
server and the client are using the same aliases, which can be implemented with
the help of a hash over the file + a token which is set upon the creation of
the file (by the client or upon deploying the object). This way, a client can
query the server if needed (e.g. GET /mg/yang.hash-alias?keys=hash,token ).
So, to sum up, the simplest solution would be to have a new resource
/mg/yang.hash-alias which is empty. A client may create the resource with a
POST operation, where the format is (YANG hash : alias).
Afterwards, the client can use the aliases to issue requests (example at the
end). This is the bare minimum, but when you have a device (client) which
manages many servers it is sufficient.
==========================
1.b)
In CoMI reserve all IDs < 256. In a different document specify, that IF you
want to use hash aliasing, then if works as following:
The value « 0 » can be used for information on the aliases:
1) GET /mg/0 -> Resource not found -> aliases are unsupported feature, don’t
even try (*standard behavior* - nothing to change to a normal CoMI object, as
the resource ID 0 is reserved and is inexistent in the YANG hash table).
2) GET /mg/0 -> {} -> aliases are supported, but no alias configuration in
place (alternatively, this could return some other info, but the source_uri key
is empty
3) GET /mg/0 ->
3.1) {"source_uri" : "/mg/yang.hash-alias", "hash": 0x127804bc}
3.2) {"source_uri" : "example.com/mg/yang.hash-alias", "hash": 0x127804bc}
This can actually be achieved in the following way:
In CoMI:
1) add YANG hash compression to section 5.4. (minor change)
2) add a statement that hash values 0-255 inclusive are reserved
3) eventually, mention as a guidance that these could be used for hash
aliasing
Afterwards, either add a new document describing hash aliasing, or add a small
section, which defines:
1) Data ID = 0 is reserved for the alias file.
2) IDs 1-255 inclusive are mapped to YANG hashes according to the alias 0
As an alternative, the /mg/0 could directly be the alias file, but I like this
to a smaller extent, as a GET /mg/0 will fetch the whole mapping, and not just
the ‘yes, aliases are in place, and we’re talking the same thing’.
==========================
I’d go with 1.b) but have no strong objections to the rest.
Best,
Alexander
REQ: POST /mg/0 <http://example.com/mg/0> (<- this one actually creates a
file on the server, e.g. /mg/yang.hash-aliases)
(Content-Format: application/cbor)
{
0x387804eb: 1,
0x15370408: 2
}
RES: 2.01 Created
REQ: GET example.com/mg/C <http://example.com/mg/C>
(Content-Format: application/cbor)
RES: 2.05 Content (Content-Format: application/cbor)
{
0x2 : "2014-10-26T12:16:51Z",
}
>
> Peter
>
> Alexander Pelov schreef op 2015-06-04 09:02:
>> Hi Andy,
>>> Le 3 juin 2015 à 23:37, Andy Bierman <[email protected]> a écrit :
>>> On Wed, Jun 3, 2015 at 12:48 PM, Michel Veillette
>>> <[email protected]> wrote:
>>>> About "I would prefer if hash collisions in a single module were not
>>>> allowed."
>>>> Agreed but how this is implemented.
>>> By checking the YANG hashes before it is published,
>>> similar to checking the YANG syntax with a YANG compiler.
>>>> About "But these numbers do not exist in any modules and so they cannot be
>>>> relied on in CoMI.
>>>> I understand but I assume that a module can be re-published with the added
>>>> statement when require. If a CoMI implementer try to implement a module
>>>> containing a clash, he can look for a version of this module containing
>>>> the special YANG statement. The likelihood of such clash is extremely low
>>>> and such fallback solution won’t be required often. However, the RFC will
>>>> specify how to resolve such situation if this ever happen. This statement
>>>> can also be used to address the need for data node identifier compactness
>>>> as mentioned by Alexander in its original email.
>>> I think the ad-hoc id assignments are pointless because they are not
>>> universal. The only thing that can be counted on is the module name
>>> and the XPath absolute path expression of each object.
>> Having a way of re-mapping (or aliasing) an ID to a
>> locally-understood/negotiated ID is really important in some networks
>> where the radio resources are extremely limited. If you are able to
>> use IDs 1-23 and you have URL hash compression, you save 8 bytes
>> (using standard CBOR, no changes here whatsoever). For IDs 24-63 you
>> save 7 bytes, 63-255 - 6 bytes (without doing any particular
>> treatment).
>> When using a general hash function, the probability of getting a hash
>> value < 255 is 10E-7, and choosing an URI that statically gives you
>> such ID simply sounds wrong (and unrealistic).
>> Two options are then possible here:
>> 1) Leave this out of CoMI
>> 2) Have a minor accommodation, and require no additional resources on
>> the devices
>> Both require defining hash clash handling mechanism, so here the
>> entire discussion on how to handle this is pertinent.
>> For me, 2) can be implemented in two ways:
>> 2.a) Explicit ID re-mapping (or aliasing) through a (external) file.
>> Here, it would be nice to have a way to specify the URI of the file,
>> e.g. on the server or on a remote location. Something like the mod.uri
>> .
>> 2.b) Explicit ID re-mapping (or aliasing) in the YANG module (e.g.
>> through the "id" option mentioned by Michel or maybe with an
>> extension, e.g. comi:id)
>> Note that if this is considered hash aliasing (we forget the
>> re-mapping), then the behavior can be very simple, e.g.:
>> It is up to the implementer to define the hash aliases, if any. In
>> case of clash with a YANG hash value, the alias must be ignored. In
>> case of clash with another alias, both aliases are ignored.
>> =======================
>> CoMI can decide on a small concession for aliasing uses - say that
>> YANG string hash values MUST be >= 64 (256?). The compiler can check
>> that all hashes fulfill this before publishing. This way, there is
>> even NO need to add any statement about alias clashes. It would be up
>> to the implementer to chose the appropriate way of aliasing (if any).
>> Aliases would be local in scope.
>> What do you think about it?
>> =======================
>> Best,
>> Alexander
>>>> About "In NETCONF, modules tend to use augment a lot"
>>>> In my document " draft-vanderstok-core-comi-06 - MV.docx ", I propose a
>>>> new definition to address this.
>>> What is it?
>>> IMO you should post comments to the mailing list not markup
>>> to a word document. It is too hard to follow.
>>>> Module Context: A module context is composed of all data nodes,
>>>> notifications and RPCs defined in a YANG module and included sub-modules
>>>> or added to them using the augment YANG statement.
>>> Almost -- this is defined in RFC 6020 -- the augmenting nodes are
>>> not part of the augmented module. They will be returned in
>>> NETCONF and RESTCONF retrieval operations for an ancestor node
>>> but they are not part of the augmented module. Only objects
>>> defined in the module namespace are part of the module.
>>> Andy
>>>> Section 4.1.3.4 show an example of include, import and augment.
>>>> Michel Veillette
>>>> System Architecture Director
>>>> Trilliant Inc.
>>>> Tel: 450-375-0556 ext. 237
>>>> [email protected]
>>>> www.trilliantinc.com
>>>> -----Original Message-----
>>>> From: Andy Bierman [mailto:[email protected]]
>>>> Sent: 3 juin 2015 14:54
>>>> To: Michel Veillette
>>>> Cc: Alexander Pelov; [email protected]; [email protected]
>>>> Subject: Re: [core] CoMI remote server clash file and YANG hash in URL
>>>> On Wed, Jun 3, 2015 at 11:16 AM, Michel Veillette
>>>> <[email protected]> wrote:
>>>>> Hi Andy
>>>>> Couple of questions about your proposed solution.
>>>>> === Question #1, New YANG statement
>>>>> Your solution works only if hash clashes within a module are resolved at
>>>>> design time. If not, the error message will return the following
>>>>> information which is not sufficient to resolve which data node is which.
>>>>> { "1234": [
>>>>> "module-A":"4567",
>>>>> "module-A":"6523"
>>>>> ]
>>>>> }
>>>> The original proposal had a 3-tuple { module-name, path-expr, new-hash }
>>>> We discussed some optimizations in the 6TISCH call to reduce the size of
>>>> the path-expr and still not require the client to store all the path
>>>> strings, such as store a few bytes from the path strings like 6th and 11th
>>>> bytes (Avoid the common ietf- prefix for many module names).
>>>> This is not deterministic. Only the full path string is fool-proof.
>>>> I would prefer if hash collisions in a single module were not allowed.
>>>> They need to be resolved at design time. If this is not possible, then a
>>>> standard rehash algorithm should be used so the collision is resolved in a
>>>> predictable way. e.g., try appending "_", then try "__", etc.
>>>> The result would be that hash collisions in a single module do not occur
>>>> in CoMI.
>>>>> Is it possible/acceptable to request a new YANG statement to manually
>>>>> assign a ID to a data node?
>>>>> Same question for a module ID?
>>>>> For example (see "id 16" and "id 25" bellow) :
>>>>> module a {
>>>>> namespace a-ns;
>>>>> prefix a;
>>>>> id 16;
>>>>> revision 2015-01-01;
>>>>> leaf leafA1 { type string; id 25 }
>>>>> container containerA {
>>>>> leaf leafA2 { type string}
>>>>> }
>>>>> }
>>>> You could have an extension statement do anything you want:
>>>> comi:id 16;
>>>> But these numbers do not exist any any modules and so they cannot be
>>>> relied on in CoMI.
>>>>> === Question #2, Introduction of an optional query parameter used to
>>>>> target a specific module
>>>>> For nodes / networks which can afford a slitely larger payload (IEEE
>>>>> 802.15.4g for example) but want to avoid the processing and storage
>>>>> associated with the porposed error message, are you opposed to the
>>>>> support of an optional CoAP query parameter use carry the module name or
>>>>> module ID ?
>>>>> For example:
>>>>> GET example.com/Y8d7s?select= ietf-6tisch-mac
>>>>> === Question #3, Use of the select query parameter instead of the URI
>>>>> Are you oposed to the use of the select query parameter instead of the
>>>>> URI to identify data node(s)
>>>> Not really.
>>>> I don't know how mixed the modules will get in CoMI.
>>>> In NETCONF, modules tend to use augment a lot, so the nodes from
>>>> 'module-A' may be nested nodes.
>>>> Retrieving just the augmenting nodes and ignoring the augmented nodes
>>>> would likely be operationally useless.
>>>> e.g., when augmenting a list, the key leafs are needed to identify the
>>>> instance.
>>>> I would like to keep CoMI extremely simple.
>>>> More bells and whistles means increased
>>>> memory and CPU requirements. I am glad CORE and 6TISCH WGs are so
>>>> concerned about constrained resources.
>>>> Only the most important bells and whistles will get standardized.
>>>>> For example:
>>>>> GET example.com/mg?select=Y8d7s
>>>>> GET example.com/mg?select=Y8d7s,NKHaA
>>>>> GET example.com/mg?select=ietf-6tisch-mac(Y8d7s,NKHaA)
>>>>> === Question #4, Compressing YANG hash base64 representation
>>>>> Are you oposed to the compression of the base64 representation as propsed
>>>>> by Alexander.
>>>> No. Optimizations that reduce the encoding size are important.
>>>> I want to use standard CBOR. No special hacks just for CoMI.
>>>>> For example:
>>>>> GET example.com/mg?select=7s
>>>>> Instead of
>>>>> GET example.com/mg?select= AAA7s
>>>>> Michel Veillette
>>>>> System Architecture Director
>>>>> Trilliant Inc.
>>>>> Tel: 450-375-0556 ext. 237
>>>>> [email protected]
>>>>> www.trilliantinc.com
>>>> Andy
>>>>> -----Original Message-----
>>>>> From: Andy Bierman [mailto:[email protected]]
>>>>> Sent: 3 juin 2015 13:09
>>>>> To: Michel Veillette
>>>>> Cc: Alexander Pelov; [email protected]; [email protected]
>>>>> Subject: Re: [core] CoMI remote server clash file and YANG hash in URL
>>>>> On Wed, Jun 3, 2015 at 9:05 AM, Michel Veillette
>>>>> <[email protected]> wrote:
>>>>>> Hi Alexander
>>>>>> At the last 6TiSCH bi-weekly call, 4 solutions have been presented to
>>>>>> the hash clashes problem. Two based on a rehash table and two based on a
>>>>>> new CoAP query parameter carrying the module name or registered module
>>>>>> ID. This query parameter is used to select unambiguously the targeted
>>>>>> data node(s) assuming that any hash collision within a module have been
>>>>>> resolved at design time. You can find this presentation in attachment.
>>>>>> Both of your proposals make sense to me but I still hope that the clash
>>>>>> file can be avoided or be optional.
>>>>> I agree we should not need the client to retrieve the rehash info before
>>>>> using a server.
>>>>> The CoMI authors have discussed a solution which I will try to write up
>>>>> today where a special "hash-clash" error is returned by CoMI if a rehash
>>>>> is needed.
>>>>> 1) client requests module-A object with hash "1234";
>>>>> there is no extended-name/local-name,
>>>>> just the YANG hash, used in any request
>>>>> 2) if '1234' has a collision in this server then a special error code is
>>>>> returned;
>>>>> The return payload has the rehash info [old-hash, { module-name,
>>>>> new-hash }+]
>>>>> { "1234": [
>>>>> "module-A":"4567",
>>>>> "module-X":"6523"
>>>>> ]
>>>>> }
>>>>> 3) the client gets the error info, and knows which module its "1234" is
>>>>> from.
>>>>> It finds the correct module name in the array and replaces its
>>>>> old-hash
>>>>> with the new-hash.
>>>>> 4) client re-sends the request, using hash "4567" instead of "1234"
>>>>> The rehash table goes away, and only this error info is used instead.
>>>>>> Michel Veillette
>>>>>> System Architecture Director
>>>>>> Trilliant Inc.
>>>>>> Tel: 450-375-0556 ext. 237
>>>>>> [email protected]
>>>>>> www.trilliantinc.com
>>>>> Andy
>>>>>> -----Original Message-----
>>>>>> From: core [mailto:[email protected]] On Behalf Of Alexander
>>>>>> Pelov
>>>>>> Sent: 3 juin 2015 11:01
>>>>>> To: [email protected]; [email protected]
>>>>>> Subject: [core] CoMI remote server clash file and YANG hash in URL
>>>>>> Hello everyone,
>>>>>> I would like to discuss with you to ideas/proposals in this mail, namely:
>>>>>> 1) Caching the yang clash file on a remote server
>>>>>> 2) Compressing the URL representation of YANG hashes
>>>>>> 1) Caching the yang clash file on a remote server Even though hash
>>>>>> collisions should happen rarely (if ever), there may be cases when it
>>>>>> could be inevitable, and as such, it may be interesting to cache the
>>>>>> clash_file somewhere on a remote server to avoid sending it over the
>>>>>> constrained wireless interface. It could be used for other cases, where
>>>>>> some frequently used resources are re-hashed for performance purposes.
>>>>>> Maybe it would be interesting to allow the server to specify a remote
>>>>>> location for the clash file? For static collisions or optimizations it
>>>>>> could be that the YANG definition includes alternative hash values,
>>>>>> which could be used in addition to the normal ones.
>>>>>> 2) Compressing the URL representation of YANG hashes Another point
>>>>>> (which is unrelated), is to elide the leading "A"s from the base64
>>>>>> representation of the hash in the URL (which corresponds to eliding
>>>>>> the leading zeroes from the hash). As specified in section 5.4. a
>>>>>> hash value of 0x0000007e will be encoded as AAAA_, but if the leading
>>>>>> A’s are elided it will be simply _
>>>>>> This could pose problems only if in the future the size of the hash
>>>>>> values is increased from 30 bits to some other (higher) value.
>>>>>> This, however, could be very useful when having to frequently access
>>>>>> the same resource. In such case, the server/network operator may
>>>>>> decide to re-hash the URI to a shorter version, e.g. the
>>>>>> /sys:system-state/sys:clock/sys:current-datetime could be remapped
>>>>>> from 0x15370408 and VNwQI to 0x1 and _
>>>>>> Best,
>>>>>> Alex
>>>>>> PS.
>>>>>> A typo (?), which may be already corrected, in the example of mod.uri on
>>>>>> Page 26: the two responses have different scheme "mod.uri" and "moduri"
>>>>>> _______________________________________________
>>>>>> core mailing list
>>>>>> [email protected]
>>>>>> https://www.ietf.org/mailman/listinfo/core
>>>>>> _______________________________________________
>>>>>> core mailing list
>>>>>> [email protected]
>>>>>> https://www.ietf.org/mailman/listinfo/core
>> _______________________________________________
>> core mailing list
>> [email protected]
>> https://www.ietf.org/mailman/listinfo/core
_______________________________________________
6tisch mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/6tisch
