Thanks for keeping the discussion going, Avia. Yeah, I do not think that my interpretation is rock solid at all and it's definitely forced. It's not hard to find examples in the spec that contradict my interpretation, but also there are others that contradict the opposite. I think we can all agree at least that the TOSCA spec writers did not pin down this topic well enough so it's necessary for us to fill in the gaps. I'm still sure that my interpretation is the most inherently correct because it preserves the OOP base contract. If we allow ad-hoc inputs we are making it that much harder to inherit existing node types while expecting their existing implementations to continue working. If we allow ah-hoc inputs, why not go ahead and allow a complete free-for-all here and allow dynamic monkey patching of derived types? (This is actually what the NFV csd04 profile is trying to do: "deprecate" parts of tosca.nodes.Compute, which of course will break tooling that relies on that contract.) The whole point of having type derivation in TOSCA seems to be to allow reuse of tools, and I insist that is especially true for operation inputs.
Your idea of supporting a "cfg" (or similar) input in ARIA is something we considered. (I don't think it's something we will have in the ARIA profile, because we need it supported by *all* operations, including the normative ones, and including custom ones created by template writers: so it would be have to be hardcoded support in the ARIA codebase.) However, the problem with this idea is that non-ARIA TOSCA parsers will fail. The whole idea of using the "dependencies" hack was to allow such templates to at least parse successfully with other parsers. Their are pros and cons to each approach, though, and I'm open to reopening this. By the way, the next commit to ARIA adds a configuration option for the parser: whether to support "ad-hoc inputs" or not. The default is to allow them: something that as you know I am very reluctant to do, but feel we have no choice at present. But, to do this properly, I think we need to also find a way to preserve the type in intrinsic functions, as mentioned in this discussion. It would still leave cases where it's possible to have type-less inputs, but I guess that's just how it is right now. [as for the side note: there are *lots* of bugs in the included TOSCA examples, including outright typos, so it's hard to consider them as canonical...] On Sat, Nov 4, 2017 at 9:26 AM, Avia Efrat <[email protected]> wrote: > I know this is an old thread, but since this is an important issue (and > I'm doing a review of old and interesting mails), I thought I'll take a > shot at a reply. > > I'm not relying on one example in the spec. actually, we can see that > inputs are defined under the normative "Standard" and "Configure" > interfaces and their operations many time throughout the spec. Isn't the > fact that inputs are assigned to interfaces and operations that do not have > the corresponding definition, in addition to the sentence from [3.5.13.3], > in addition to the interfaces being are the only entities in TOSCA that do > not have explicit distinction between definition and assignment enough to > suggest that this may have been intentional? > > These usages of untyped inputs are still there in v1.2, and unfortunately, > artifact definitions still do not have a properties-like field. > > I liked your idea suggesting that "untyped" inputs get their type > definition from the intrinsic functions, that seems to settle between the > unusualness of interface definitions, and the fact that TOSCA gives us the > notion that assignment should be derived from a definition. This could also > be a (temporary?) solution to the lack of "operation type" concept, since > it does not seem to solved in v1.2. > > In addition, I agree that this does not solve the problem of your proposed > inputs/meta inputs (inputs/configurations) distinction. Just a thought, we > could decide that an input with a special name such as "cfg" could be a map > containing "configurations", while other inputs are treated as regular > inputs. This could part of the ARIA profile. > > [side note: > You said: > And indeed this is exactly what we see example 2.14.2, too, where we see > property assignments. But this example snippet doesn't prove much of > anything, because we don't see the node type defined, just used here. > The "wp_db_port" input could/should be declared there. > But in fact this type is defined in [8.3 > <http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile- > YAML/v1.0/os/TOSCA-Simple-Profile-YAML-v1.0-os.html#_Toc471725259>], > The section for non-normative types used for examples throughout the spec. > This section also exist in v1.2, with no changes to the aforementioned node > type] > > On Mon, Sep 11, 2017 at 8:01 PM, Tal Liron <[email protected]> wrote: > > > The single sentence you mention in the 3.5.13.3 is the only place that > > *might* be implying that ad hoc, type-less input assignments are allowed, > > but I actually think it could have a different reading. What i meant is: > > "...that do not necessarily have a property definition defined in its > > corresponding *interface* type." I think what is implied is what I > mention > > in level #2: a node template could have assignments that are not in the > > interface type, *but obviously they have to be defined in the node type*. > > Obvious, I say, because they have to be defined somewhere: an undefined > > property assignment makes little sense in TOSCA. So let me rewrite that > > sentence with more clarity, without (I think) changing its original > > meaning: > > > > "Template authors MAY provide property assignments on operation inputs on > > templates that do not necessarily have a property definition defined in > its > > corresponding interface type by adding an input definition to the node > > type." > > > > To be more honest here: when I say "I think what is implied" I have to > > wonder if the author was completely clear on what was implied. Like many > > parts of spec, there's a lot of theory that clashes in practice. We are > > confused, but it could be that the author was confused as well. By > > implementing the spec we have a responsibility to pin down what's fuzzy. > > > > The "partial" differentiation as you call it in 3.5.13.1 is what I meant: > > in some places you use property definitions, in other assignments. The > > language is very clear that you use definitions with types and > assignments > > with templates. They didn't explicitly have an "operation assignment" > > grammar, but it's clear that there are two different grammars. When I say > > "operation assignment", it's a shorthand for the kind of operation > > definition that you find in templates which uses property assignments. > This > > is consistent with everything else in TOSCA. > > > > And indeed this is exactly what we see example 2.14.2, too, where we see > > property assignments. But this example snippet doesn't prove much of > > anything, because we don't see the node type defined, just used here. > > The "wp_db_port" > > input could/should be declared there. The fact that ARIA accepts this has > > always been very problematic to me: we here have an input that is > untyped. > > I consider this a bug. As we discussed, this particular example *might* > > make a bit of sense because we are using an intrinsic function, which > could > > carry the type with it... but nowhere in the spec is any mechanism like > > that explained. Many of the examples in the TOSCA spec are wrong or > > incomplete, too, so I wouldn't be shocked if this one is wrong as well. > > > > Anyway, none of this addresses the core issue in my opinion, which I keep > > returning to: these kinds of parameters we add here -- SSH user, > password, > > etc. -- are meant as *directives to our execution plugin*, conceptually > > very different from operation inputs. Mixing them together is confusing > > (what do you do with name overlaps?) as well as a major security concern. > > These two kinds of values simply should not be mixed together. Indeed, in > > TOSCA 1.2 they might end up being artifact properties. > > > > > > On Mon, Sep 11, 2017 at 11:25 AM, Avia Efrat <[email protected]> wrote: > > > > > In contrary to most of the TOSCA entities, TOSCA does not differentiate > > > between 'definition' and 'assignment' in the context of operations. > There > > > are only "operation definitions" [3.5.13]. Logically, there is a > partial > > > differentiation [3.5.13.1], where inputs in node type operations are > > > expected to be property definitions, and inputs in node template > > operation > > > are expected to be property assignments (which are actual value > > > assignments). Both of these options are listed under "operation > > > definition". > > > > > > Under [3.5.13.3] it is explicitly stated that > > > "Template authors MAY provide property *assignments* on operation > inputs > > on > > > *templates* that do not necessarily have a property definition defined > in > > > its corresponding type." (my emphasis) > > > > > > That is, from this paragraph, I think that it is clear that you can > > define > > > operation inputs in node templates (level #3) without them being > defined > > in > > > the node template's type. In fact, [2.14.2] is an example of doing just > > > that. And out parser treats such a syntax as valid, see: > > > https://github.com/apache/incubator-ariatosca/blob/ > > > master/tests/parser/test_tosca_simple_v1_0/test_end2end.py#L73 > > > > > > Me thinking it was a good idea to construct TOSCA that way is another > > thing > > > =) > > > > > > On Mon, Sep 11, 2017 at 6:35 PM, Tal Liron <[email protected]> wrote: > > > > > > > Feel free to change the wiki, Ran, to whatever name you find > > appropriate. > > > > > > > > I think what Avia discovered is not new to us and actually doesn't > > solve > > > > the problem, unfortunately. Let me go over what is clearly allowed > and > > > not > > > > allowed in TOSCA, confusing because there are a few levels of > > inheritance > > > > here. > > > > > > > > 1. Interface types. Obviously, you are allowed to inherit an > interface > > > type > > > > and add or override inputs. (ARIA insists that overridden input types > > be > > > > derived from what it is they are overriding, too, to keep the OO > > contract > > > > intact.) > > > > > > > > 2. Node types. In the "interfaces" section you can define several > > > > interfaces of various types. Here, again, TOSCA lets you add/override > > > > inputs. Though note here that the line of inheritance is quite > complex: > > > you > > > > can override inputs from the interface type, but *also* from the > parent > > > > node type. (ARIA here has to do some work to make sure that you are > > doing > > > > it all OK and not breaking the OO contract, it's a rather complex > part > > of > > > > the parser code.) > > > > > > > > I think the above is what Avia discovered. However, the third level > is > > > > locked to us: > > > > > > > > 3. Node templates. Unfortunately, here we can not add inputs ad hoc. > > The > > > > "interfaces" section here is not the same DSL format as those above: > it > > > is > > > > operation *assignments* rather than operation *definitions*. When you > > > > assign input values here, they are validated against the defined > types. > > > It > > > > would make no sense in TOSCA to just assign values without a type. > > > > > > > > So, because we can't add inputs at level #3, we still have a problem: > > we > > > > would have to derive new node types for every type of execution. SSH > > > would > > > > require its own node types, Juju would require its own node types, > > Puppet > > > > would require its own node types, etc. And that's for *all* your node > > > > types. This seems extremely un-scalable. > > > > > > > > But also, as I try to explain in the wiki page, I insist that these > > kinds > > > > of configuration parameters are essentially not the same as operation > > > > inputs. They are not meant to be used by the operation itself > (script, > > > > charm, recipe, etc.), rather by the mechanism that executes the > > operation > > > > (SSH, Juju, Puppet, etc.). Especially I point out the security hole: > > you > > > > don't want an SSH password exposed and sent over the wire to the > script > > > > itself. It is simply not an input. > > > > > > > > By the way, it seems that there's some acknowledgement by other folk > in > > > > OASIS about this gap in TOSCA, and there's an interest to use > artifact > > > > types in TOSCA 1.2 as a way to solve this problem. I don't think > it's a > > > bit > > > > awkward, but at least there will be a standard solution. > > > > > > > > > > > > On Sun, Sep 10, 2017 at 2:13 AM, Ran Ziv <[email protected]> wrote: > > > > > > > > > Avia's mentioned at one point that we might have misunderstood the > > spec > > > > at > > > > > this section, and that in fact it can be possible to pass arbitrary > > > > inputs > > > > > into operations regardless of the interface definition - which > would > > > mean > > > > > this notion of "configuration" might be unnecessary. > > > > > > > > > > Also, note that the doc page is talking about "executors", which is > > > > > confusing as that's a different concept in ARIA (see the base > > executor > > > > > class); Supposedly up until now we've simply called these > "operation > > > > > plugins". > > > > > > > > > > > > > > > On Sat, Sep 9, 2017 at 1:43 AM, Tal Liron <[email protected]> wrote: > > > > > > > > > > > Yes: > > > > > > https://cwiki.apache.org/confluence/display/ARIATOSCA/ > > > > > > Execution+Configuration > > > > > > > > > > > > On Fri, Sep 8, 2017 at 4:45 PM, DeWayne Filppi < > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > I see in the examples a list of dependencies for scripts in > > > > operations, > > > > > > for > > > > > > > example: > > > > > > > > > > > > > > implementation: > > > > > > > primary: scripts/configure.sh > > > > > > > dependencies: > > > > > > > - "ssh.user > { get_input: ssh_username }" > > > > > > > - "ssh.key_filename > { get_input: > > private_key_path > > > > }" > > > > > > > - "ssh.address > { get_attribute: [ virtual_ip, > > > > > > > floating_ip_address ] }" > > > > > > > > > > > > > > The spec seems to indicate that the dependency list is for > > > resources > > > > > that > > > > > > > need to be made available so the main script can be run. > Clearly > > > > that > > > > > > > isn't the case in the example. Is this '>' syntax documented > > > > anywhere? > > > > > > > > > > > > > > > > > > > > > > > > > > > >
