On Sat, Jul 31, 2021 at 9:26 AM Balázs Lengyel <balazs.lengyel= [email protected]> wrote:
> Hello, > > Sorry for going back to the basics, but IMHO it is needed here. So as I > see understand the problem: > > The purpose of the draft and some principles should be clearly stated. In > my view: > > > > *P1) “There is a need to validate configuration data against data created > by the system.”* > > > > We also have some principles: > > *P2) We want the running datastore to contain exactly what the operator > has written there* > > > > From this it flows that > > *P3) the running/intended configuration MAY become invalid if the > system-data changes. * > > But > > *P4) rfc8342#section-5.3.4 MUST always be valid* > > > > P3 and P4 contradict each other. The only way to resolve this IMHO is to > say that system-data changes only at upgrade. If the upgrade can’t end up > with a valid configuration (after whatever processing) then it MUST fail. > > > > IMHO > > If we move the system-data into a separate datastore that does not help. > When I update the running configuration the validation will depend on > system-data either in the system or in the intended datastore. Debugging a > configuration distributed over multiple datastores is difficult. > > IMHO we should just introduce a new datatype beside config=true and > config=false, let’s call it system-data or read-only-config=true. > > System-data would be the same type as config=true except it is not > writable by Netconf/Restconf/CLI/SNMP, etc. only by the system itself. > > This would allow us to define and populate system-data in both > running/intended datastores. It would be visible in the same context. > > System-data would be static, the operator would not be able to modify it, > so impact on netconf/Restconf operations would be trivial. > > > I agree a new datastore will just add complexity without any value. Your solution approach is better, but I think it would require a new YANG version to allow config node XPath to reference non-config nodes. Another solution is to model the referenced node as config=true, but setup automatic NACM rules so no user editing is allowed. This works well for setting up an initial config that gets saved and not changed unless a reset is done. What if <intended> is what is NV-stored? When that occurs, the config changes from system to user config. Routers have been saving the combined config for decades. IMO the standards intentionally avoid discussing the conversion of a datastore to/from NV-storage. As I see it this is the same problem that we discussed in > https://github.com/netmod-wg/yang-next/issues/41. > > > > I know this is a radical change, but I think using a new YANG extension to > create a read-only config=true datatype is a much cleaner solution. One > which some companies have already implemented. > +1 Actually, separate running and system would be a radical change, not this. Cleaner and less disruptive. If I misunderstood your intent, then sorry. > > > > Regards Balazs > > > Andy > *From:* netmod <[email protected]> *On Behalf Of *maqiufang (A) > *Sent:* 2021. július 31., szombat 14:33 > *To:* Kent Watsen <[email protected]>; [email protected] > *Subject:* Re: [netmod] system configuration sync mechanism > > > > Hi, Kent, > > Thanks for helping me revive this thread, which is exactly what I want to > do.:) > > There was not a very fully discussion due to time constraints, but we did > see some valuable points here, thank you everyone for sharing your views. > > > > Regarding option2, I am still unsure how will things go if there is no > <intended>(I think it was raised by Balazs, hopefully Balazs can also add > something here)? Should <system> be implemented along with <intended>? > > > > Option 3 is still unclear, e.g., whether the <system> is copied into > <running> automatically or manually? If auto-copy is not a good idea > because it violates the definition of <running>, whether manual-copy is > performed towards part or all of the system configurations created in > <system>? > > Should we copy the entire <system> into <running>? Or should there be as > few system configuration data items in <running> as possible? > > Anyway, I agree that option3 may still incur a failed validation of > <running> when the operators reference the system configuration which is > produced through the expansion of the system-defined templates. > > If the existing mechanism(e.g., edit-config)is sufficient to define > referenced system data item in <running>, it seems that the flow marked in > option3 from <system> to <running> can be removed, then it looks no > difference between option1 and option3. > > > > Best Regards, > > Qiufang Ma > > > > *From:* netmod [mailto:[email protected] <[email protected]>] *On > Behalf Of *Kent Watsen > *Sent:* Thursday, July 29, 2021 1:09 AM > *To:* [email protected] > *Subject:* Re: [netmod] system configuration sync mechanism > > > > WG, > > > > Regarding yesterday’s <system> datastore presentation, there seemed to be > support for "Option #2”, which is to have <system> merge into <intended>. > > > > It was noted that this then would mean that client-validation of <running> > would necessitate understanding how the merge works, to expand templates, > resolve leafrefs, etc. > > > > My thoughts are, so? > > > > Firstly, a client that doesn’t understand that there may be some <system> > defined configuration will, for the most part, be none the wiser. The > client *will* discover <system> configuration in <operational>, but this is > already the case today. One new thing is that <operational> should use > “origin:system” for configuration originating from the <system> datastore. > This last point might surprise clients…as the definition of “with-origin” > doesn’t state that clients must ignore any unrecognized “origin” > identities: https://datatracker.ietf.org/doc/html/rfc8527#section-3.2.2. > > > > Secondly, no shared object defined in <system> will be activated until > client-supplied config references it. But any client able to do this > already knows how <system> merges into <intended> and is accounting for it. > > > > Thoughts? > > > > Kent > > > > > > On Jul 16, 2021, at 6:24 AM, maqiufang (A) <[email protected]> wrote: > > > > Hi, Kent, > > Please see my reply inline. > > > > *From:* Kent Watsen [mailto:[email protected] <[email protected]>] > *Sent:* Friday, July 16, 2021 2:55 AM > *To:* maqiufang (A) <[email protected]> > *Cc:* [email protected] > *Subject:* Re: [netmod] system configuration sync mechanism > > > > Hi Qiufang, > > > > * [snip]* > > The question is if the server implementation prunes dangling/unused > objects when <intended> is applied, updating <operational>. My assumption > is that the server will discard any object that doesn’t actually impact the > running configuration of the system (i.e., values are consumed by the > underlying operating system, drivers, etc.). Thusly, it is my opinion that > only the referenced objects are applied. Hence why, to answer your last > question, I wrote that these configurations (manufacturer-defined objects) > are not applied immediately but only after they are referenced. Makes > sense? > > *[Qiufang Ma] Yes, try to sum up our discussion about the categories of > the system configuration:* > > · *Physical-resource-dependent--> whether this sort of system > configuration exists in <system> dependents on if the physical resource is > present(e.g., physical interface).* > > · *Physical-resource-independent-->which is provided by the device > system* > > o *Further classification from the perspective of “applied” > time(dependents on whether the system configuration impacts the running of > the system)* > > § *Config that is applied immediately(e.g., the loopback, the predefined > minimum length of password…)* > > § *Config that is applied only after being referenced by other > configs(e.g, definitions for applications ftp/tftp…)* > > o *Further classification from the perspective of generation time* > > § *Config that is generated unconditionally at each boot time(e.g, > loopback, predefined minimum length of password, ftp/tftp…)* > > § *Config that is generated conditionally during the device > running(e.g., system-generated local-port and remote-port for a new > established BGP connection)* > > > > Keep in mind that what is described above is just one aspect of what can > be in <system>. In addition to defining reference-able objects, <system> > can also define/apply configuration immediately (e.g., the loopback > interface). That is, configuration not does not have to be referenced in > order to become activated. > > *[Qiufang Ma] Noted.* > > > > > > > > Note that, <running> by itself would not pass validation, due to missing > leafrefs. Thankfully, NMDA never says that validation runs on <running>. > But once <running> and <system> have been merged, to become <intended>, the > result does pass validation. > > [Qiufang Ma] The referenced instance must also exist for the data to be > valid since the require-instance defaults to true if not present. Is this > what you had in your mind? Yes, NMDA says that it is <intended> which is > subject to validation. But I also notice that In section 5.1.3 of the > NMDA:”<running> MUST always be a valid configuration data tree, as defined > in Section 8.1 of [RFC7950]. ” So my thought here is that <running> should > also conform to the YANG model constraints and that’s to say, a referenced > system-defined data item should also exists in <running>. > > Therefore, if system configurations do not exist in <running>, they still > need to be configured in <running> manually in order for being referenced. > In this case, the original purpose of predefining some system configurations > for user convenience is lost. This is the reason why we would like to define > some mechanism here to synchronize <system> into <running>. > > > > > > I see in RFC 8342 "<running> MUST always be a valid configuration data > tree, as defined in Section 8.1 of [RFC7950]”. > > > > But the question remains if it is possible for the system is able to > validate <running> without, e.g., expanding templates. There may be a > 'leafref' or ‘must’ expression somewhere that will fail because the > evaluation occurs without expanding a template that supplies the missing > parts. > > *[Qiufang Ma] On condition that <running> should be valid, the operators > will need to retrieve from the <intended> or <operational> to get the > template-expanded configurations and then create them in the <running>, > right? * > > *My feeling is that it loses the meaning of predefining and seems no > differences between operator-defined configurations if operators have to > create system configures in <running> before they use them. So I am > beginning to think, if it’s possible to expand the system-defined template > during the copying between <system> and <running>?* > > > > If this draft “updates” RFC 8342 (NMDA), then it can supply a clarifying > statement about what it means that "<running> MUST always be a valid > configuration data tree”. Either that, or an Errata if it’s determined > that the statement isn’t correct. > > > > You make a good technical point, but I think that we should *want* to > avoid having to copy <system> (or <operational>) configuration into > <running> if we can avoid it. Agreed? > > *[Qiufang Ma] tend to agree. Maybe we should try to avoid it, unless we > have to.* > > > > FWIW, also in RFC 8342, Section 5.1.4.: > > > > <intended> is tightly coupled to <running>. Whenever data is written > > to <running>, the server MUST also immediately update and validate > > <intended>. > > > > <intended> MAY also be updated independently of <running> if the > > effect of a configuration transformation changes, but <intended> MUST > > always be a valid configuration data tree, as defined in Section 8.1 > <https://datatracker.ietf.org/doc/html/rfc7950#section-8.1> > > of [RFC7950] <https://datatracker.ietf.org/doc/html/rfc7950#section-8.1>. > > > > > > > > > > > I am wondering if these configuration will present in the <operational> > > (which contains all the configuration actually used by the device) before > > they’re referenced. > > > > I think that it would depend in the specific server’s behavior, regarding if > *unused* predefine objects are present in <operational>. Certainly the > unused objects would not have to be present in <operational>. If I were > implementing the server, the unused objects would NOT be present in > <operational>. > > [Qiufang Ma] Yes, if the predefined system configurations is unused, then I > also tend to agree that they would not be present in <operational> but may > also depend on the vendor implementation. > > > > > > Yup, this is the same conclusion is in my response above. > > *[Qiufang Ma] :)* > > > > > > > It would be good if we could determine if there are any other > > "resource-independent” configuration categories here. > > > [Qiufang Ma] Do you think there exists conditional system configuration (if > > the preceding configurations you mentioned above is not)? For example, if > > SSH is enabled on a device, SSH-related keys are automatically generated. > > Such configurations are generated at the moment when a special > > functionality is enabled. > > > > I’m unsure what you mean in general by "conditional configuration”, but I can > speak to your specific example. Though I must preface my comments that I > imagine there are a number of ways servers might go about enabling `sshd`. > What follows is my personal view, forged by being around systems for awhile > ;) > > > > In general: > > > > - `sshd` is NOT enabled by default. > > - `sshd` is enabled via a configuration knob. > > - the SSH host key is dynamically generated the first time `sshd` is > enabled. > > - the SSH host key itself is in <operational> (not <running>) > > > > This view is consistent with the first paragraph in Section 3 of the > “keystore” draft (reproduced below): > > > > 3. Support for Built-in Keys > > > > In some implementations, a server may support built-in keys. > Built- > > in keys MAY be set during the manufacturing process or be > dynamically > > generated the first time the server is booted or a particular > service > > (e.g., SSH) is enabled. > > > > As a closing thought, this model (which I stated upfront may not be > universal) would have no presence-in or interaction-with <system>…though, > perhaps, there may be some predefined values for what key-algorithms and/or > key-lengths to use when generating the SSH host key... > > [Qiufang Ma] By “conditional system configuration”, I was meaning some of the > system configurations are not generated immediately after the device is > powered on. Instead, they are generated when a specific condition is > satisfied during the device running(e.g., a functionality is enabled due to > some client configurations). I am not sure if it really exists, maybe not, > just try to explore the possibilities of various resource-independent system > configurations.:) > > > > To this point I agree..as does RFC 8342 (NMDA), Section 5.3.3.: > > > > Sometimes, resources are controlled by the device and the > > corresponding system-controlled data appears in (and disappears from) > > <operational> dynamically. If a system-controlled resource has > > matching configuration in <intended> when it appears, the system will > > try to apply the configuration; this causes the configuration to > > appear in <operational> eventually (if application of the > > configuration was successful). > > *[Qiufang Ma] See above, as I summarized as the system configurations that is > generated conditionally during the device running.* > > > > Firstly, I again have to preface my comment that there are likely many ways > that templating mechanisms can be defined. But, in general, once a > templating mechanism has been defined, then it stands to reason that > templates could be defined either in <running> (by operators) or in <system> > (by the manufacturer). In one implementation I’m familiar with, the > templates are objects that are referenced/parameterized by other parts of the > configuration. (Same as with the predefined objects discussion above.) > > > > To answer your questions: > > > > 1) Yes, it is my opinion that *activated* templates in <system> will be > expanded and present in <intended>. > > > > 2) I would never suggest that the system-defined templates are present in > <running>, though they may be referenced/parameterized by config in <running>. > > > > 3) if a config-template is configured in <running> (i.e., it is > operator-defined) then, yes, the expanded configuration in <intended> is > "client configuration” (note, "client configuration” is not a formal term). > That said, it seems fair to say that a template defined in <system> and then > referenced by "client configuration” in <running> is also expanded as "client > configuration” in <intended>. > > > > 4) I don’t not understand your last sentence, that the expansion of <system> > templates are only present in <operational>. Maybe you’re saying something > subtle, e.g., that servers currently don’t support GET on <intended>. But, > in theory, the expansion of <system> templates should (IMO) be present in > <intended>, so that they may be subject to validation. Of course, all the > <intended> configuration (whether originating in <running> or <system>) that > is successfully “applied” will also be present in <operational>. > > [Qiufang Ma] Assume that there is no <system> and this work, the expansion of > system templates are only present in <operational>. Because this is > compatible with system configuration definition in NMDA. > > But if system configurations are only present in <operational>, the > predefined system configurations still need to be retrieved and created into > <running> explicitly when being referenced. I think we’ve reached an > agreement on the need for <system> to exist, and our main point of > disagreement is whether <system> should be copied into <running>. Your point > is that being merged into <intended> is enough to make sure a success > validation. But my understanding is that the referenced system configuration > data item must also exist in the <running> to obey the model constraints. > > > > Yes, I believe that you provided an accurate description of the difference > in our opinions. Per my earlier response, you make a valid technical > point, my goal is to waive that interpretation to the side so that a > simpler solution can emerge. It would be good to get other opinions on > list, otherwise we’ll take it into the meeting. > > *[Qiufang Ma] OK. Hopefully someone else would share some opinions here. > Otherwise let’s take this into the IETF meeting.* > > > > [BTW, in keeping with this thread moving from the NETCONF to the NETMOD > mailing lists, would it make sense to move the IETF 111 presentation slot > from NETCONF to NETMOD too? I think it does and, further, it would help > with scheduling (NETCONF is over, NETMOD is under). Would you be okay with > this? AD Rob and the NETCONF chairs discussed this morning, and think > it's okay, but would still need to confirm with the NETMOD chairs.] > > *[Qiufang Ma] I am happy with the proposal, if it’s also okay for NETMOD > chairs:-). I have sent an email to the NETMOD chairs to request to move > this presentation slot from NETCONF to NETMOD. * > > *A new version of the draft will also be submitted to NETMOD when the > draft-submitting window reopens.* > > > > <big snip> > > > I’m beginning to think that: > > > · auto-copying into <running> is likely never a good idea, because > > it violates the definition of <running> > > > [Qiufang Ma] I am quite aware that different datastores in NMDA represents > > different views of data nodes. And <running> represents a configuration > > datastore holding the current configuration of the device. > > > Should we consider system configuration also be part of current > > configuration of the device? From my perspective, the difference between > > system configuration and client-configuration lies only in who provides it. > > > > <running> holds the current *operator-specified* configuration of the device. > System-provided configuration is NOT specified by operators (though > system-defined objects may be referenced by operator-specified config in > <running>). I believe that this arrangement is consistent with the > definition of <running>. Agreed? > > [Qiufang Ma] Yes. Actually we are not trying to violate the principles of > NMDA and the definition of <running>. The issue we try to resolve here is > that system configurations cannot be used(referenced or overwritten) by the > operators directly and need to be created into <running> explicitly. This > actually loses the meaning of “predefining and bringing convenience”. If > auto-copying is not a good idea, what do you think about defining an RPC > operation for the operators to do the copy(which is also what Rob suggests at > the meeting)? > > > > If we have to copy into <running>, then I think that I agree an RPC > (<edit-config>?) would be better. > > *[Qiufang Ma] From my perspective, <edit-config> is feasible but not > efficient because operators still need to retrieve <system>/<operational> > firstly. If we could define a RPC to copy the entire <system> into > <running>, it seems more convenient for operators. However, some system > configurations which are not going to be referenced or modified may also be > copied into <running>. I don't have a strong feeling about which one is > preferred. Anyway, we need to figure out whether it would be fine for > <running> to missing referenced system configurations.* > > > > You mention “overwritten” by the operators? Why wouldn’t the operators > just define their own? For instance, if they don’t like the vendor’s > “vendor-foobar” object, they could copy/paste/edit their own “my-foobar” > object with the values needed, yes? > > *[Qiufang Ma] Yes, defining their own would be okay. By overwriting, I > mean sometimes the operators would like to modify the specific system > configuration, e.g., the MTU value of a specified interface(identified by > its name).* > > *If the operators want to modify the system configurations, there is no > way but redefine them in <running>.* > > > > > > *Best Regards,* > > *Qiufang Ma* > > > > > > > > > > > · having in <operational> doesn’t make sense, since the tweaks > > wouldn’t go thru <running> --> <intended> validation. > > > > > > I’m wondering if a model like below would work for everyone - thoughts? > > > [Qiufang Ma] <intended> represents the configuration after all > > configuration transformations to <running> have been performed, so I think > > it is only coupled to <running>. > > > Anyway, the <system> should also interacts with <operational>. Agreed? > > > > I don’t agree that <intended> must only be coupled to <running>. > Specifically, I think that it is okay (compatible with NMDA) to define a > <system> that also impacts <intended>. This is the only (IMO) sane > approach, as it enables the combination <running> + <system> to be validated. > > [Qiufang Ma] Please see above. If <running> is OK to miss referenced system > configuration, your proposal makes sense to me. > > > > Ack. > > > > > > * Best Regards,* > > *Qiufang Ma * > > > > > > Kent // contributor > > > _______________________________________________ > netmod mailing list > [email protected] > https://www.ietf.org/mailman/listinfo/netmod >
_______________________________________________ netmod mailing list [email protected] https://www.ietf.org/mailman/listinfo/netmod
