On 30/01/2019 17:31, Andy Bierman wrote:


On Wed, Jan 30, 2019 at 8:02 AM Robert Wilton <[email protected] <mailto:[email protected]>> wrote:


    On 30/01/2019 15:16, Andy Bierman wrote:


    On Wed, Jan 30, 2019 at 4:19 AM Robert Wilton <[email protected]
    <mailto:[email protected]>> wrote:

        Hi Andy,

        Thanks for the comments.

        On 30/01/2019 01:22, Andy Bierman wrote:
        Hi,

        I originally brought up this issue in July 2015
        https://datatracker.ietf.org/doc/draft-bierman-netmod-yang-package/

        Yes.

        The solution I propose is different in the sense that it uses
        YANG instance data to define YANG packages rather than new
        YANG keywords.   I believe that this should make it a lower
        cost solution to define and implement.



    I think yangvalidator.org <http://yangvalidator.org> has a better
    solution that does not change YANG conformance.

    Do you mean that we can just use zip files with the list of modules?


I don't care about the solution details yet. They are 2nd order problems.

Conformance means "what modules are required to be implemented together".
It is not clear that this problem can be solved.  The augment-stmt defines implicit multi-module conformance.  I am not convinced that the extra work of defining package conformance
is worth it.

So, I'm not proposing backing any sort of package conformance into the language at all.  A package is just metadata that defines that a set of modules, at particular revisions/versions, work together and can represent part of a YANG schema.

This is equivalent to
 - how a zip file of YANG modules provided to yangvalidator would work.
 - getting the contents of YANG library from a server (but a YANG packages soln can also work offline)  - fetching the modules from YANG catalog (if they have been labelled appropriately), although I'm not convinced that this universally works today.

But in terms of the usability of YANG, I don't think that doing conformance only at the module level is really sufficient. Clients need to be coding against sets of modules at particular versions that are known to work together, and known that multiple server vendors will implement.

A pick and mix appropriate to module revisions doesn't seem to help anyone.



The issue of what modules does vendor A implement is not a conformance problem. It is just more metadata and YANG Catalog is focused on providing that data.

Does YANG catalog indicate the set of IETF modules that I would need to implement L2VPN services on a device?

Module tags could be used to do this (another packaging solution), but this would cause a proliferation of tags when it comes to versioning, since I don't think that you can cleanly bake semver into module tags.



    I don't really see how this helps.

    Consider:
    - server vendor A, implements some subset of the OpenConfig YANG
    modules, each at a particular version, along with some deviations
    and vendor augmentations.
    - server vendor B, implements the same subset of the OpenConfig
    YANG modules, but at different versions, along with some
    deviations and vendor augmentations.
    - server vendor C, implements a slightly different set of the
    OpenConfig YANG modules, but at different versions, along with
    some deviations and vendor augmentations.

    As a client, how do I know what module versions to code against,
    when I want to work with devices provided by all three vendors?



The vendors publish their implementation details on yangcatalog and you get the info
and see what modules are in common.

There are only market requirements determining what group of modules has to exist in an implementation.  It is not clear to me that formalizing these requirements is something the industry will do effectively.  Module tags already provide a way
to conceptually group modules together.

Seems like every vendor has openconfig, foo-openconfig, and foo-openconfig-deviations so that there are no agreed upon subsets. Even if openconfig had properly documented subsets, would your client even be able to code to it (ignoring add-ons and deviations).

I think that answer will converge on yes, I don't know how long it will take.  It would probably be better if at the time that protocol specifications are written, that the authors of the specifications also write the YANG modules to manage them at the same time.


    I might be wrong, but I think that the OC solution is use git
    tags, so they tag sets of modules that are expected to work
    together and also to provide a linear release history of their
    sets of modules.  So, if everyone implements the module versions
    associated with a git tag then it should convert a two dimensional
    problem of module revisions into a linear problem.  The YANG
    packages draft is aiming to provide a solution to this problem
    that doesn't require the use of git, or sending zip files of
    modules around.


    At the moment, it seems that everyone is doing this in different ways:
     - Yumawork customers/servers use zip files of modules for
    conformance.


Not sure what this means.
Actually the server libraries can be loaded and unloaded.
Module can be standalone libraries or grouped as bundles.
But this seems like an implementation detail, unrelated to conformance.


     - OpenConfig client/server implementations use git tags, or git
    refpoints.
     - Cisco customers use the contents of directories on github
    YangModels.

    Finally, this draft doesn't change YANG conformance, it just
    expresses it in what is intended to be a simpler way.


It adds another conformance system to maintain.
The language only recognizes module to module interfaces, not package to package.

I propose that at the language level conformance is at the module level (modulo import-by-version).


That adds more complexity. It doesn't take away any complexity.

It is meant to be a simpler way of packaging up, and trying to control and manage the complexity that already exists today.

E.g. I could give a YANG compiler the name, version, and location of a package and tell it to build the entire schema associated with that package.

In a somewhat similar way, when I write code, my build file specifies which libraries I depend on, and their versions, but I can leave it to the build tool to determine what those libraries themselves depend on and recursively pull in all the dependencies.



If there was a standard to load and unload modular functionality at boot-time or run-time, then I could see why there is a need to have a standard to define YANG packages.

I agree that this is another example of where they could be useful.

Thanks,
Rob



    Thanks,
    Rob



Andy



    Andy


        I don't think the WG ever agreed on the problem that needs
        to be solved,
        and that is still the case.

        That wasn't quite my impression.  I also think that folks
        were busy focusing on other WG activity and didn't
        necessarily have the time to concentrate on this.

        My draft was aiming on solving two broad problems:

            The main goals of YANG package definitions include, but are not
            restricted to:

            o  To act as a simplified YANG conformance mechanism.  Rather than
               conformance being performed against a set of individual YANG
               module revisions, conformance could also be more simply stated in
               terms of YANG packages, with a set modifications (e.g. additional
               modules, deviations, or features).

            o  To allow YANG datastore schema to be specified in a more concise
               way rather than having to list all modules and revisions.  YANG
               package definitions can be defined in documents that can be
               referenced by a URL rather than requiring explicit lists of
               modules to be shared between client and server.  Hence, a YANG
               package must contain sufficient information to allow a client or
               server to precisely construct the schema associated with the
               package.




        In reality each server has 1 package -- its entire library.

        This doesn't apply to all servers.  For a long time, as a
        vendor, we have had separate packages that can be
        independently installed, and which extend the management
        model to cover the new functionality.  E.g. BNG functionality
        could be in a separate, independently installable, package on
        top of the base router functionality.

        For a Linux server, the manageability interface will depend
        on what applications have been installed.


        The SEMVER work shows
        that vendors are treating platforms as independent release
        trains, and not really
        developing loadable packages.

        This depends on the vendor.  The YANG versioning work is
        trying to find a solution that works across the industry.  I
        believe that the versioning requirements are different for
        standards developed modules, vs industry developed modules,
        vs vendor modules.



        I think YANG 1.2 improvements for conformance (e.g.,
        YANG-redirects, SEMVER import)
        and  the YANG Catalog can solve the module compatibility
        issues. It is more of a documentation
        problem than a standards problem.

        Having a standard YANG module that can be used to define
        packages is something this is useful and should be
        standardized.  I believe that this is better than each vendor
        coming up with their own solution for this problem.

        Thanks,
        Rob




        Andy




        On Tue, Jan 29, 2019 at 4:55 PM Sterne, Jason (Nokia -
        CA/Ottawa) <[email protected]
        <mailto:[email protected]>> wrote:

            Thanks Rob. Please see inline.

            Jason

            *From:*Robert Wilton <[email protected]
            <mailto:[email protected]>>
            *Sent:* Thursday, January 24, 2019 1:16 PM
            *To:* Sterne, Jason (Nokia - CA/Ottawa)
            <[email protected]
            <mailto:[email protected]>>; [email protected]
            <mailto:[email protected]>
            *Subject:* Re: initial comments on
            draft-rwilton-netmod-yang-packages

            Hi Jason,

            Thanks for the review and comments.

            I've put some responses inline ...

            On 24/01/2019 14:56, Sterne, Jason (Nokia - CA/Ottawa)
            wrote:

                Hi guys,

                I've gotten most of the way through the draft and
                have some initial comments. I haven't digested the
                section 10 open issues yet or the examples.

                Section 5 mentions the following:

                YANG library is augmented to allow servers to report
                the packages

                that they implement and to associate those packages
                back to

                particular datastore schema.

                Does the combination of this draft and rfc7895bis
                somehow allow the same package to be advertised in 2
                different datastores, but with different deviations
                in each datastore? I'm thinking of a case, for
                example, where a package is fully supported in the
                running but the package minus a few modules (or
                parts of modules) is supported in the operational
                datastore. There seems to be a 1:1 relationship
                between package and rfc7895bis schema.

            So, the intention is no, not directly.

            My aim here is that <running> would implement package
            "foo", and <operational> would implement package
            "modified-foo". Package "modified-foo" would import
            package "foo" and also specify the set of modules that
            contain the deviations "foo".

            I didn't want a server to be able to see that I
            implement package "foo", but then I have all these
            deviations that change its behavior.  Instead, it is
            really implementing a different package that is based on
            "foo".

                The packages draft doesn't include any specific
                leaf-list for deviations. Section 7.2 mentions that
                deviations could be expressed by including modules
                that happen to contain deviations. That seems a bit
                inconsistent with rfc7895bis that has a specific
                leaf-list of deviations (and NETCONF hello that
                specifically explicitly labels deviation modules).

            I'm conflicted on this one.  I don't really like the
            deviation list in YANG library because I regard it as a
            duplicate source of information, and then there is a
            question of which source of data do you trust.  E.g. do
            you process a deviation in a module that is not listed
            in the deviations module list?

            */[>>JTS: ] Good point. I suppose this issue applies
            today already. i.e. what if one of the modules
            advertised in the <hello> is a module of deviations
            (without having been referenced by another module as a
            deviation module)./*

                Section 5.1 says the package must be referentially
                complete. I can see the advantages of that although
                wondering if that might limit flexibility of
                partitioning modules into packages. I could imagine
                use cases for dividing a large set of modules into a
                few packages that might rev independently but can
                still all work together (especially if they rev in a
                bc manner). But maybe that just starts to introduce
                too much complexity?

            Yes, having partial packages may be useful.  Perhaps
            just adding a leaf to indicate whether a package is
            referentially complete could be the answer here.

                I didn't understand this part of section 5.1. Can
                you maybe illustrate with an example?

                The version/revision of a module listed in the
                package module list

                supercedes any version/revision of the module listed
                in a imported

                package module list.  This allows a package to
                resolve any

                conflicting implemented module versions/revisions in
                imported

                packages.

            Probably best to see example B.3. in the appendix
            because it exactly illustrates this point.

            Basically:
            1) Packages must explicitly list all versions of all
            modules they define/import.
            2) If two imported packages define different versions of
            modules, then the package that is importing them needs a
            way to define which version to use.
            3) A package needs a way to override the version of
            module specified in an imported package.

            */[>>JTS: ] Thx. That example does help. I suppose the
            designer of the package needs to carefully check that
            the version they select can be successfully used by all
            the modules in the package. /*

            */I think there is a minor typo in example B.3.  The
            example-3-pkg is importing "/* */example-import-1" but I
            believe you meant "/* */example-import-1-pkg" (and some
            for import-2)./*

                It might be a good idea to add a parent-version to
                the package module (to allow tracking lineage of
                packages).

            Agreed, or maybe allowing a revision history like
            modules. Not sure which is better here. Packages could
            get a lot of updates, and a long revision history would
            not be helpful at all.

            */[>>JTS: ] I think a minimum of just specifying the
            direct parent is enough to build the full tree of
            lineage. We don't need a long history of N revisions./*

                I like the use of groupings. That allows a manager
                to use this as a building block to compose a model
                that has a list of packages.

            OK.

                Having a global list of mandatory features (vs
                having the mandatory feature a per-module list)
                means inventing the new <module-name>:<feature>
                format. Should we instead somehow put the mandatory
                features against each module of the package?

            Perhaps.  My thinking here was to have the list of
            features high up and very easy to find/parse.

                The location leaf is a uri but then the description
                says it must be a url (where the model can be
                retrieved). I do like that the namespace is separate
                from the location, but maybe we should make location
                a url type?

            Yes, I was thinking that is should be a URL.

                Do we need a namespace for package names in the model?

            I had them in an earlier version, but I took them out,
            because I wasn't sure that they are really useful/required.

            Defining a format to make package names themselves
            globally unique might be sufficient.

            */[>>JTS: ] I'm OK with that. It is similar to how we're
            finding that it is useful that YANG module names are
            globally unique (i.e. by naming with ietf-xxxx or
            companyabc-xxx)./*

                In 7.3 we only reference module-sets and not
                modules. So the grouping of modules into sets and
                packages must be the same?

            Not necessarily.

            I am trying to reuse the module-set definitions as much
            as possible (to avoid duplication). One issue here is
            that module-sets are combined then all the modules must
            not overlap, which doesn't make the mapping to
            module-sets quite so clean.

                A schema can only have a single package. I think
                that works but it means a server would advertise
                multiple schemas if it wants to support multiple
                packages. I'm not sure if there are some downsides
                to that (it just surprised me).

            My aim here was:
             - multiple packages are advertised in yang-library/packages
             - datastores only report that they "implement" one [top
            level] package version.  [The package itself might
            import other packages.]

            If we do package selection, then for a given YANG client
            session, and the version of YANG library
            available/reported by that session, it would appear as
            if the server only implements one top level package for
            a datastore. Different clients choosing different
            versions would see slightly different output depending
            on which package version they had selected to use.

            Thanks again for the review and the comments!

            Rob

                Jason

                *From:*netmod <[email protected]>
                <mailto:[email protected]> *On Behalf Of
                *Robert Wilton
                *Sent:* Thursday, December 20, 2018 12:45 PM
                *To:* [email protected] <mailto:[email protected]>
                *Subject:* [netmod] YANG Packages

                Hi,

                I've written up an ID for a potential solution for
                YANG packages using instance data:

                Abstract

                   This document defines YANG packages, an
                organizational structure

                   holding a set of related YANG modules, that can
                be used to simplify

                   the conformance and sharing of YANG schema.  It
                describes how YANG

                   instance data documents are used to define YANG
                packages, and how the

                   YANG library information published by a server
                can be augmented with

                   additional packaging related information.

                
https://datatracker.ietf.org/doc/draft-rwilton-netmod-yang-packages/

                Potentially this work may be of use as part of the
                YANG versioning design team work. In addition, if
                the WG likes this approach of defining YANG
                packages, then it might also be useful to bind a
                schema to a YANG instance data document.

                Some questions for members of the WG:

                1) Do members of the WG agree that YANG packages is
                something that needs to be solved?

                2) Is the approach in this draft of defining these
                as instance data documents a good starting point?

                3) This approach augments YANG library-bis, reusing
                module-sets, but not replacing the way that modules
                are reported in YANG library-bis. Is this the right
                approach? This approach tries to allow module-sets
                to be reused for both schema and packages, but the
                YANG library-bis rules for combining module-sets
                (i.e. no conflicts) may make this harder to really
                reuse the module-sets for both purposes.

                Of course, any other comments or feedback is welcome
                and appreciated.

                Thanks,
                Rob

            _______________________________________________
            netmod mailing list
            [email protected] <mailto:[email protected]>
            https://www.ietf.org/mailman/listinfo/netmod

_______________________________________________
netmod mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/netmod

Reply via email to